Loading...
 
Features / Usability

Features / Usability

Forum List Topic List

Forums » Features / Usability » batch upload large (~100 GB) files
 
Thread actions
PDF Print this page
  • Print all pages

    • batch upload large (~100 GB) files

    Posted by Theobio posts: 8

    Hi,

    I am using a wiki for archiving purposes and need to upload large files, up to several 100 GB. This should be possible through batch upload, which works fine up to 20 GB, on Tiki version 9.6. Larger files (~100 GB) result in an Internal server error message, suggesting a timeout somewhere. The file has actually been moved from the upload dir to the tiki dir used for file gallery storage with its name encrypted as usual, but this entry does not appear in the tiki_files table in the database. Questions:

    1. what setting in php could affect this? php max_execution_time is 180, and storage quota on the server are ample.

    2. tiki documentation for version 10 indicates that "It is now possible to integrate very large files into the Tiki File Gallery using Batch Upload. See Batch Upload for details." However, there is no definition of "very large files" there or in the Batch Upload referred to, so would an upgraded tiki version be able to handle `100 GB files correctly?

    Thanks!
    Joke

    Reads: 2773
    Link

    Posted by Theobio posts: 8

    Hi,

    I increased all php variables relevant for upload limitations to large (250 GB) or unlimited when available (on a local install, so no server-side limitations). I still have problems with large files. Batch upload more or less works up to 20 GB, meaning that the files are stored in the target dir & included in the file gallery, but they display & download incorrectly when size > 2 GB. Either size displays as 2.0 GB (on live server installation), or as the actual but negative value (local test installation). In both cases, downloading the file will not work, downloads are 0 GB in all browsers (IE, Firefox, Chrome) I tested.
    Actual file size = 2,265,686,016 bytes
    Value in tiki file gallery after upload: -2 029 284 587 B
    The uploaded file is ok, it can be downloaded again directly through ftp & unzips and reads correctly.

    I noticed that newer tiki versions (10 and up) document "It is now possible to integrate very large files into the Tiki File Gallery using Batch Upload.
    See Batch Upload for details." but it never defines how large "very large" is.

    Has anyone been able to store large files, 10-200 GB, on tiki 10, 11 or 12 through batch upload yet?

    Cheers, Joke

    Link