Reading and organising files and folders dropped on a web page from the file system

In this blog post we will demonstrate how to read and organise files dragged from the file system and dropped into some dropzone on a web page. This article does not focus on the actual file uploading process, but rather on how to read the files and organise them in a form that is suitable and easy to upload. You can checkout the working code for this article at https://github.com/n05la3/parallel-task-runner

Creating the HTML and attaching the necessary events

First, we need to create the HTML for the dropzone and style it.

<div class="drop-container" id="dropzone">
  <div>Drop files/folders here to upload</div>
</div>

/* Simple styling assuming nothing else is on the page */
.drop-container {
  height: 50vh;
  width: 50vw;
  border: 1px dashed #ccc;

  display: flex;
  flex-direction: column;
  justify-content: center;
  align-items: center;
}

Next, we need to attach the necessary events to the dropzone. We listen to the dragover event in order to prevent the default behaviour of the browser which is to try opening the file.

const dropzone = document.getElementById('dropzone');

dropzone.addEventListener('dragover', (event) => {
  event.preventDefault();
});
dropzone.addEventListener('drop', handleDrop);

Implementing the handleDrop function

We can now implement the TypeScript function handleDrop which will be called when a file is dropped on the dropzone. It’s implementation looks something like this:

async function handleDrop(event: DragEvent) {
  // Do not attempt to handle the drop if it is not a file/folder drop or if there are no files/folders
  if (
    !event.dataTransfer ||
    event.dataTransfer?.items.length === 0 ||
    event.dataTransfer?.items[0].kind !== 'file'
  ) {
    console.log('Not a file drop');
    return;
  }

  const filesFound = Array.from(event.dataTransfer.items).flatMap((item) => {
    // @ts-ignore - getAsEntry is not available yet, webkitGetAsEntry may get renamed to getAsEntry.
    // So we are coding defensively here for that. See https://developer.mozilla.org/en-US/docs/Web/API/DataTransferItem/webkitGetAsEntry
    const file = item.getAsEntry ? item.getAsEntry() : item.webkitGetAsEntry();

    try {
      if (!file) {
        return [];
      }

      return file;
    } catch (error) {
      console.error('At least one file could not be read', error);
      return [];
    }
  });

  console.log('filesFound', filesFound);

  const filesRead = await extractFilesDropped(filesFound);
  console.log('filesRead', filesRead);

  const reorderedFiles = await reorderFiles(filesRead);
  console.log('reorderedFiles', reorderedFiles);
}

handleDrop starts by ensuring the event.dataTransfer object contains some items and at least the first item in the list is a file. This is to ensure we are not dealing with a non-file drop. We then parse all the files to extract the FileSystemFileEntry or FileSystemDirectoryEntry using webkitGetAsEntry() which returns null if the entry is not a file.

The FileSystemFileEntry or FileSystemDirectoryEntry in filesFound contains the entries at the root of the dropped content. These are not the actual files, we will have to read the files. This is what the extractFilesDropped function is used for. Here is its implementation:

type FileOrDirectory = Pick<
  FileSystemEntry,
  'name' | 'isDirectory' | 'isFile' | 'fullPath'
> & {
  file?: File;
  children?: FileOrDirectory[];
};

function isFile(fileEntry: FileSystemEntry): fileEntry is FileSystemFileEntry {
  return fileEntry.isFile;
}

async function extractFilesDropped(files: FileSystemEntry[]) {
  const filesFound: FileOrDirectory[] = [];

  for (const fileEntry of files) {
    if (isFile(fileEntry)) {
      filesFound.push({
        name: fileEntry.name,
        isFile: true,
        isDirectory: false,
        fullPath: fileEntry.fullPath,
        file: await readFileFromFileSystem(fileEntry),
      });

      continue;
    }

    const directoryReader = (
      fileEntry as FileSystemDirectoryEntry
    ).createReader();

    const filesInDirectory: FileSystemEntry[] = [];
    let fileBatch: FileSystemEntry[] = [];

    // There's a browser limitation that prevents reading more than 100 files at a time, so we need to read them in batches
    // until there are no more files to read
    do {
      fileBatch = await readDirectoryEntries(directoryReader);
      filesInDirectory.push(...fileBatch);
    } while (fileBatch.length > 0);

    filesFound.push({
      name: fileEntry.name,
      isFile: false,
      isDirectory: true,
      fullPath: fileEntry.fullPath,
      children: await extractFilesDropped(filesInDirectory),
      isRoot,
    });
  }

  return filesFound;
}

The directory and files are read differently. The FileSystemFileEntry contains a file method which returns a File object that can be used to read data from the file represented by the FileSystemFileEntry. The file method accepts an error and success callback where its success callback contains the entry read. We will use a Promise so that we can await it. Here’s it’s implementation:

function readFileFromFileSystem(fileEntry: FileSystemFileEntry): Promise<File> {
  return new Promise((resolve, reject) => {
    fileEntry.file(
      (entry) => {
        resolve(entry);
      },
      (error) => {
        reject(error);
      }
    );
  });
}

The FileSystemDirectoryEntry contains a createReader() method which returns a FileSystemDirectoryReader object used in reading the directory entries. Here’s the implementation of the method which accepts the FileSystemDirectoryReader object and is responsible for reading the directory entries:

function readDirectoryEntries(
  reader: FileSystemDirectoryReader
): Promise<FileSystemEntry[]> {
  return new Promise((resolve, reject) => {
    reader.readEntries(
      (entries) => {
        resolve(entries);
      },
      (error) => {
        reject(error);
      }
    );
  });
}

We could just call this method and pass to it the FileSystemDirectoryReader object returned by the createReader() method and get the directory entries but some browsers have a limitation that prevents reading more than 100 files at a time, so we need to read them in batches. We use the do...while loop to read the files in batches until there are no more files to read.

At this point, we should have all the entries from the files dropped on the dropzone. Because extractFilesDropped is recursive, it will read all the directories in the dropped content and all their subdirectories. Subdirectories at each recursive level are placed in the children property of the parent directory.

Having this level of nestedness may not be desired especially if one just needs to upload the files and not care about the folder structure. We could achieve this by modifying the extractFilesDropped function and making it return a flat array of FileOrDirectory objects.

As a bonus, we will add a method to reorder the files in a flat array of FileOrDirectory such that files/subdirectories in a directory come after the directory itself. This means if we were to upload the files one at a time to respect the structure, then we are assured that the directory will be created before all its children.

function reorderFiles(
  files: FileOrDirectory[],
  filesInFileSystem: FileOrDirectory[]
) {
  for (const entry of files) {
    if (entry.isFile) {
      filesInFileSystem.push(entry);
    }
  }
  for (const entry of files) {
    if (entry.isDirectory) {
      filesInFileSystem.push(entry);
    }
    if (entry.children && entry.children.length > 0) {
      reorderFiles(entry.children, filesInFileSystem);
    }
  }

  return filesInFileSystem;
}

// The reverse is needed because the directories before the files at each level. If we want to start uploading the files at the level before the folders, then we need to reverse the order
const reorderedFiles = reorderFiles(fileEntries).reverse();

The sortFiles function is recursive and will call itself on the children property of each directory entry. Before it does this, it is added to the filesInFileSystem array ensuring that it is placed in the array before any entry in it.

What happens if we drop 300 files and we need to upload all of them to a server? We probably would not want to upload all 300 in one go. We can continuously upload about 5 files at a time and continuously pick up a file once one is completed ensuring that we are always uploading about 5 at a time but not going above. In this post, we build an upload processor to do this.

If you need any help with this tutorial or if you would like to request a development consultancy you can Contact us or you can check our Github Sponsorship page directly.