Commons File Upload contains a severe memory leak

Do you use Commons File Upload library in your application? Do you use DiskFileItemFactory for storing big files to a temporary disk storage? Do you use FileCleaningTracker to get rid of unused temporary files as it is recommended in documentation?

If so you probably have a memory leak in your application you don't know about.

Stored files are meant to be cleared when there is no reference to the FileItem instance created by this library. Or at least this is stated in the documentation:

"Such temporary files are deleted automatically, if they are no longer used (more precisely, if the corresponding instance of java.io.File is garbage collected. This is done silently by the org.apache.commons.io.FileCleaner class, which starts a reaper thread."

Whole logic is written in a very clever way excluding a little bug in DiskFileItemFactory class on this spot:

public FileItem createItem(String fieldName, String contentType,
            boolean isFormField, String fileName) {
   DiskFileItem result = new DiskFileItem(fieldName, contentType,
                isFormField, fileName, sizeThreshold, repository);
   FileCleaningTracker tracker = getFileCleaningTracker();
   if (tracker != null) {
      tracker.track(result.getTempFile(), this);
   }
   return result;
}

On the place where tracker registers new file to track instance of this is passed there as a "marker" reference. This obviously means DiskFileItemFactory that is usually kept as a long living instance in the application. This marker is then propagated to this place:

private static final class Tracker extends PhantomReference {
   private final String path;
   private final FileDeleteStrategy deleteStrategy;
   Tracker(String path, FileDeleteStrategy deleteStrategy, Object marker, ReferenceQueue queue) {
      super(marker, queue);
      this.path = path;
      this.deleteStrategy = (deleteStrategy == null ? FileDeleteStrategy.NORMAL : deleteStrategy);
   }
   public boolean delete() {
      return deleteStrategy.deleteQuietly(new File(path));
   }
}

And is used as a reference of the PhantomReference. This reference is then monitored for garbage collection instead FileItem instance as it should be according to documentation. Because DiskFileItemFactory is long living thing trackers in FileCleaningTracker only get collected and NEVER get cleaned!

We found this issue when experiencing OutOfMemory problems on our server that was undergoing a penetration test that tried to upload huge amount of files through open web form. Using Eclipse MAT we found out that there are more than 350 thousands of tracker objects inside FileCleaningTracker that are not cleared even after original requests and sessions were all dropped.

Please vote for this issue if you want to see it resolved.

Meanwhile you can build your own version of the Commons File Upload with this patch.