MOSS 2007 - Uploading and Indexing Large Files:

A client has reported an issue that users were not able to upload files larger than 50MB file which we can resolve easily by updating Web Application settings; however, users were getting Timeout exceptions as well. Furthermore, business told us that business requirement is to be able to upload up to 100MB files.

There are few simple configurations setting those needs to do in order to upload large files in to SharePoint document libraries….

SharePoint Configurations:

• Increase the maximum upload size
  The maximum size for uploading files is set to 50 MB for a Web Application. If you need to be able to upload larger files, you can change this setting to any value up to 2 GB (2047 MB). However, as this is web application level setting so even if you have multiple site collections, this change will replicate to all site collections.

  1- Go to Start –> All Programs -> Administrative Tools –> SharePoint 3.0 Central Administration -> Applications Management.
  2- Go to Web application general settings under SharePoint Web Application Management section.
  3- Look for Maximum Upload Size and update that field as required up to 2 GB (2047MB)
  4- Click OK to save Settings.

• Increase the default chunk size for large files
  The chunk size is not related to the maximum upload file size. The chunk size simply specifies the amount of data that can be read from a file at one time. By default, the large-file-chunk-size property is set to 5 MB. If you notice performance or scale problems on the client or server, then you may need to tune this setting to get the performance you are targeting.
  Note: If you raise the chunk size too high, the files might use up too much front-end memory and you may need to lower this setting.
  
  The large–file–chunk–size property must be set from the command line. This property is configured for a server or server farm, and cannot be configured for an individual virtual server. To set this property, use the following syntax:

  1- Open Command Prompt
  2- Navigate to 12 Hive Bin Folder (usually C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\Bin) 
  3- Run Command: Stsadm.exe –o setproperty –pn large–file–chunk–size –pv <Size in Bytes>
  4- After making a change to this property, you must restart IIS. You can restart IIS by typing iisreset on the command line.

Tune the Microsoft Internet Information Services (IIS) connection timeout setting:

By default, timeout for is set to 120 seconds (2 minutes). Depending on your maximum file size and how long it takes for the file to be uploaded, you may not need to change this setting. If, however, IIS is timing out when you upload large files, you can change this property to ensure that larger files can be uploaded successfully.

1- Go to Start –> All Programs -> Administrative Tools –> Internet Information Services (IIS) Manager.
2- Right-click the virtual server you want to configure, and then click Properties.
3- Click the Web Site tab.
4- In the Connections section, in the Connection timeout box, type the number of seconds you want IIS to wait before timing out.
5- Click OK to save settings.

Those who are working in SPS 2003 and want to make changes for web parts, please have a look to this article and find Configuring Large File Support.

Web.Config configurations:

• Update maxRequestLength and executionTimeout attributes for the httpRuntime node: 
  excecutionTimeout = <Time in Seconds>
  maxRequestLength = <File size in KB> (by default for ASP applications, its set to 4096 (4MB), but because SharePoint supports 50MB so its set to 51200)
  e.g.
  <httpRuntime executionTimeout="999999" maxRequestLength="102400" />   <!-- 102400 = 100MB -->

Indexing Large files:

If large files needs to be crawl then we need to make following changes in the Registry and central administration.

• Registry Configurations:
  By default, Search Services can crawl and filter a file with a size of up to 16 megabytes (MB). It will always crawl the first 16MB of a file. After this limit is reached, SharePoint Portal Server enters a warning in the gatherer log “The file reached the maximum download limit. Check that the full text of the document can be meaningfully crawled.”
  To increase the limit of 16 MB, you must add in the registry new entry MaxDownloadSize.  To do this, follow these steps:
  

  1- Start Registry Editor (Start –> Run –> Regedit).
  2- Locate the following key in the registry: HKEY_LOCAL_MACHINE -> SOFTWARE –> Microsoft –> Office Server -> 12.0 –> Search –> Global –> Gathering Manager
  3- If already MaxDownloadSize exits then move to step 5. 
  4- Right click –>  New –> DWORD Value. Name it MaxDownloadSize.
  5- Double-click on MaxDownloadSize, change the value to Decimal, and type the maximum size (in MB) for files that the gatherer downloads.
  6- Restart the server (this is an important step to do, otherwise changes will not effect).
  7- Start Full Crawl.

NOTE: Increasing the file size may cause a timeout exception because the crawler can timeout if the file takes too long to crawl/index (because of its size). To increase timeout value, follow these steps:

• Central Administration Configurations:  

  1- Go to Start –> All Programs -> Administrative Tools –> SharePoint 3.0 Central Administration -> Applications Management.
  2- In the Search section, click Manage search service
  3- On the Manage Search Service page, in the Farm-Level Search Settings section, click Farm-level search settings.
  4- In the Timeout Settings section change Connection and Request acknowledgement time. (here you need to do your math that how big files are suppose to be index and approximately how long it will take to crawl – for 100MB I haven’t change it so it’s still default 60 seconds for both).

You are all done and ready to test your scenarios :).

Enjoy!!!