Recursive file iteration in Java

In a java frameworks talk I gave recently I showed the following example for finding files recursively…

public void listFilesInDirectory(File dir) {
  File[] files = dir.listFiles();
    if (files != null) {
      for (File f : files) {
         if (f.isDirectory()) {
	    listFilesInDirectory(f);
	 }
	 else {
	    System.out.println(f.getName());
	}
     }
  }
}

… which in real projects often grows to a larger block of code. The web is full of code blocks like this for walking a directory tree. The best version I came across is this object oriented one by Torsten Curdt. Since you usually don’t want to write this yourself, I suggested in my talk to use FileUtils which makes recursive iteration much easier:

Collection jspFiles = FileUtils.listFiles(rootDirName,
                        new String[] { "jsp" }, true);

This looks concise and useful but as I tried to use it, I wasn’t too pleased with the FileUtils’ solution. Here is why:

  • The recursion is processed in one go, i.e. all results are written to a List even when using the iterateFiles method. The recursion is not processed iteratively.
  • You can not influence the directories that are searched.
  • Only files are returned, you can not search for directories.
  • The API is not very expressive (e.g. what does the “true” mean).
  • No generics (raw collection types are returned).

A Better API

Not being satisfied with the solutions I found, I “dreamed up” my own API for listing and finding files. I don’t consider it complete but for the most part I am pleased with the ease of use that the builder pattern provides. The code for this can currently be found in an unrelated goole code project. The rest of this article shows the functions that are currently supported.

Find files two ways

There are generally two ways to use the result – as interator or as list:

1. Iterate over all files in the windows directory:

for (File f : Files.find("c:\\windows")) {

}

2. Get all the files in a directory as a list of files:

List<File> allFiles = Files.find(somedir).list();

Except from the return type the second version does the same as the JDK command listFiles:

File[]  allFiles = (new File(somedir)).listFiles()

Easy recursive listing

To iterate all the files in the C:\Windows directory, you would use:

for (File f : Files.find("c:\\windows").recursive()) {

}

Note: This actually works iteratively, i.e. the recursion happens as you fetch files from the iterator. The result is not fetched into a huge list.

With a Predicate you can limit the recursion to specific directories. In this example all .svn directories within a source tree are skipped:

Predicate<File> noSvnDirs = new Predicate<File>() {
boolean apply(File file) {
return !file.getName().equals(".svn");
}
}
for (File f : Files.find("src/java/").recursive(noSvnDir)) {

}

Want Files, Directories or both?

Define if you want only files, only directories or both in your result with yield*()-Methods.

Files.find(someBaseDir).recursive().yieldFiles()  // this is the default
Files.find(someBaseDir).recursive().yieldDirectories()
Files.find(someBaseDir).recursive().yieldFilesAndDirectories()

Filtering the results

To get all textfiles within a dir use:

Files.find(dir).withExtension("txt").list();
Files.find(dir).ignoreCase().withExtension("txt").list();

You can also filter by Name, e.g. to find README files:

Files.find(dir).withName("README").list();
Files.find(dir).ignoreCase().withName("readme").list();

Note that the default matching is case sensitive. The commands caseSensitive() and ignoreCase() can be used to toggle the matching behaviour.

For special needs you can also specify a Predicate<File> to filter the resulting files.

Files.find(dir).recursive().withFilter(somePredicate).list();

Finding Directories

When looking for directories there are some special usecases that are supported, e.g. looking for directories that contain a specific file:

Files.find(dir).recursive().yieldDirectories()
               .containingFile("Thumbs.db");