File format detection

File format detection is a hard problem to solve so abydos doesn't even try to solve it. Instead a MIME type has to be provided by the user. Some types of files, especially newer ones, can be reliably identified by looking at the few first bytes of the file. This is called a magic signature.

There are two major magic signature databases which can be used for getting the MIME type of a file, libmagic and Shared MIME info. This goes a long way, but some files can't be identified this way. Sometimes because they lack an entry in any of those databases or because they lack a magic signature. Some files can still be automatically identified in other ways, such as reading last few bytes of the files or analyzing it more in depth. This can be done using GNU libextractor (with custom plugins).

But there are still some formats that somply can't be identified automatically. If they are simply dumps of the graphics RAM (quite common for older image formats) then any combination of bits is a valid file. In some cases the file size is an indication, but not a fully reliable for determination. The only thing left to do is to let the user specify the file type, perhaps by setting up a custom mapping file names to MIME types.

Using the file extension guess the file type might be useful to some extent, but it's a far from optimal solution. Some extensions obvious extensions like pic and img are used by several image formats. And extensions like bin and raw (and also img) are often used for binary images in general. Not just for graphics, but ROM images (including CD images), executables and just any kind of raw binary data. Sometimes the file size can be used to tell them apart since older graphics files are typically miniscule.