This post details how to use Python to open and read from an Ekahau Site Survey (ESS) project file. The examples in this post were created with macOS, but the concepts also apply to Windows.

warning “DISCLAIMER” The code used in this post does not perform write operations, but you should make a backup of your Ekahau project file before running any code against it. This post is educational, and is not supported by Ekahau.

Short on time? The too long, didn’t read is that an ESS project is at it’s core a ZIP file, and you can use Python’s built-in zipfile module to work it.

This ESS project is similar to how you can rename the filename extension of a Microsoft Word (.docx) or PowerPoint (.pptx) file to .zip and then extract it. You can rename the ESS extension from .esx to .zip and then open the archive. For example on macOS you could do: Open With > Other... > Archive Utility.app.

via Gfycat

In this post, I explore using Python to confirm that an ESS file is a ZIP file, and then read a file from it.

Table of Contents

Magic Bytes

Magic bytes (also referred to as file signatures) refer to a block of byte values used to designate a filetype. They’re typically used by applications to identify if a file is in the right format.

So, we know by that we can rename the extension from .esx to .zip, but how can we determine what type of file this is? We can use these “magic bytes” to determine the file signature/type.

Python-magic and libmagic

There are a few different ways to do pull this off and do file identification. We could roll our own code (more complex), or could use several different external modules (easier). This section is about identifying filetype using a package called python-magic which is a wrapper around the libmagic file identification library.

python-magic is an external dependency that we will need to install. We can get it from PyPI and install it like: pip install python-magic. Note that since it’s a libmagic wrapper you will also need to install libmagic. If you don’t install libmagic you will get an error when you try to use python-magic. To install libmagic with Homebrew: brew install libmagic.

Start a Python shell. First import the needed modules: os and magic:

>>> import os
>>> import magic

Next create a for loop. We will use it to loop over each file in the current directory. os.listdir() will give us a list containing names of the files.

>>> for file in os.listdir():

note “Note” You could also do: >>> for file in '/Users/<user>/path/here': (macOS) or >>> for file in 'C:\Users\<user>\path\here': (Windows), but we should write code that works for either and only write it once, so we are going to use os and its inner functions to access the filesystem.

Press enter and you will see the prompt change from >>> to an ellipsis .... This means you will need to indent the next command in order for Python to know it belongs to for block of code.

Using python-magic we want to get the MIME type (file signature) for a file. In this example we’re going to loop through every file in the current directory and check its MIME type.

>>> for file in os.listdir():
...     _magic = magic.from_file(file, magic.MAGIC_MIME_ENCODING)
...     print("{} type: {}".format(file, _magic))
...

note “Note” A MIME type indicates the nature and format of a file.

Now hit enter again to leave the for block of code and run the loop.

If you ran this from a directory that contains any subdirectories, the code should throw an IsADirectoryError exception:

>>> for file in os.listdir():
...     _magic = magic.from_file(file, magic.MAGIC_MIME_ENCODING)
...     print("{} {}".format(file, _magic))
...
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "/usr/local/lib/python3.7/site-packages/magic.py", line 136, in from_file
    return m.from_file(filename)
  File "/usr/local/lib/python3.7/site-packages/magic.py", line 86, in from_file
    with open(filename):
IsADirectoryError: [Errno 21] Is a directory: '.config'

Let’s handle IsADirectoryError with try and pass.

  • try specifies exception handlers and/or cleanup code for a block of code.
  • pass is a null operation. It acts as a placeholder and does nothing. We’re using it because we don’t care about doing anything with the exception, and a statement is syntactically required.
>>> for file in os.listdir():
...     try:
...         _magic = magic.from_file(file, magic.MAGIC_MIME_ENCODING)
...         print("{} {}".format(file, _magic))
...     except IsADirectoryError:
...         pass
...

Press return once more to leave the for block of code. If you opened the Python shell from your home directory, you might see some of these files in the output:

.DS_Store application/octet-stream
.bashrc text/plain
.vimrc text/plain
.hyper.js text/plain
.bash_profile text/plain
.python_history text/x-python
.gitconfig text/plain
.bash_history text/plain
.viminfo text/plain

Let’s open the shell in a directory that contains an .esx file, and type in the same code from above again. The output may look something like this:

tar-cjf.tar.gz application/x-bzip2
test.tar.xz application/x-xz
test-zip.zip application/zip
test.esx application/zip
tar-zcvf.tar.gz application/x-gzip

In my test, I included a couple different types of archives that use different compression methods. This shows that different compression methods have different file signatures (and different magic bytes).

What we care about is that both the .zip and .esx files have the same file signature. This confirms that an ESS project is stored as a common zip archive.

With that knowledge, our next step is to figure out how to work with zip archives in Python.

Leveraging the zipfile module

Python has a built-in module called zipfile to read and write ZIP files. We need at least Python version 3.2 or greater for the following code to work. This is because we will use zipfile as a context manager with a with statement (introduced in Python 3.2).

A context manager allocates and releases resources exactly where you need it. Here’s what a context manager’s syntax looks like:

with context_manager as my_object:
    my_object.do_something()
    print('done')

note “Note” my_object only exists inside the block of code below the with statement.

Before the block of code below the with context_manager as my_object: is executed, the context manager loads a few special methods, like __exit__() for later use; typically to clean up resources. For example using zipfile like with zipfile() as zip: will automatically open and close the archive without you needing to explicitly call those inner functions (.open(), and .close()).

Let’s open the Python shell in a directory that contains an ESS project file. You can change the current directory by using os.chdir('path') and use os.getcwd() to confirm the current directory changed. Or you could exit the shell, navigate to that directory, and then open the Python shell again.

Now we will import ZipFile from zipfile, and run it’s inner function .namelist() to get the archive’s members by name:

>>> from zipfile import ZipFile
>>> with ZipFile('test.esx', 'r') as zip:
...     zip.namelist()
...
['requirements.json', 'accessPoints.json', 'usageProfiles.json', 'networkCapacitySettings.json', 'deviceProfiles.json', 'attenuationAreaTypes.json', 'project.json', 'images.json', 'floorTypes.json', 'projectConfiguration.json', 'simulatedRadios.json', 'version', 'antennaTypes.json', 'wallTypes.json', 'floorPlans.json', 'applicationProfiles.json', 'image-552383b1-9e64-4a2b-85ce-7fe25b82c5d0']

note “Having trouble?” Notice the letter case of ZipFile vs zipfile

I recommend verifying if the file we’re trying to open with ZipFile is actually a ZIP file. Lucky for us, the zipfile module contains an inner function called .is_zipfile() that will check it for us. The function returns True if it is, otherwise returns False.

That means we don’t actually need to use python-magic or roll our own file identification code (that is if we don’t want to use an external dependency). If you want to learn more about rolling your own file identification code see the appendix for an example that doesn’t use python-magic or libmagic.

So, we only need to leverage zipfile.is_zipfile() to verify the file we’re opening is a zip. Once we open the zip file, we can use ZipFile.getinfo(member) to check if the member file we want to read actually exists. If the member is not there we will get a KeyError. We should handle that exception.

Here’s a code example using zipfile with some basic error handling. Here’s what it does:

  1. determines if the file is a zip file
  2. opens the file using a context manager
  3. reads and prints the names of the members contained in the zip
  4. tries to read and print the contents of a specific member

The code:

>>> import zipfile
>>> file = "test.esx"
>>> member = "version"
>>> if zipfile.is_zipfile(file):
...     with zipfile.ZipFile(file, 'r') as zip:
...         print("namelist(): {}".format(zip.namelist()))
...         try:
...             info = zip.getinfo(member)
...             data = zip.read(member)
...         except KeyError:
...             print("ERROR: did not find {} in {}".format(member, file))
...         else:
...             print("{} is {} bytes".format(info.filename, info.file_size))
...             print("data: {}".format(data))
... else:
...     print("problem with zipfile")
...
namelist(): ['accessPoints.json', 'antennaTypes.json', 'applicationProfiles.json', 'attenuationAreaTypes.json', 'deviceProfiles.json', 'floorPlans.json', 'floorTypes.json', 'image-20d26177-6624-4d8b-859a-f1fd31f77815', 'images.json', 'networkCapacitySettings.json', 'project.json', 'projectConfiguration.json', 'requirements.json', 'simulatedRadios.json', 'usageProfiles.json', 'version', 'wallTypes.json']
version is 3 bytes
data: b'2.0'

At this point it makes sense to write all this up in a file with the Python code.

Exercise 1:

  1. Write up a file with the code.
  2. Try changing the member string from version to something else from namelist(). The point of this is so you can see what block of code executes when there is a problem.
    • 2a - something else that exists.
    • 2b - something that doesn’t.
  3. Try reading from something inside of the archive and doing something with it.

Thank you for reading. If you have questions, comments, or corrections feel free to reach out.

Appendix

A simplified alternative to python-magic and libmagic

Here is an simplified example that checks if a file is a zip without using external dependencies.

>>> MAGIC_BYTES = {"\x50\x4b\x03\x04": "zip and formats based on it e.g. pptx/xlsx/docx"}
>>> MAX_LENGTH = max(len(x) for x in MAGIC_BYTES)
>>> def file_type(filename):
...     """"check magic bytes"""
...     # encoding='latin-1' is to deal with invalid utf-8
...     with open(filename, encoding="latin-1") as file:
...         file_start = file.read(MAX_LENGTH)
...     for magicbytes, filetype in MAGIC_BYTES.items():
...         if file_start.startswith(magicbytes):
...             return filetype
...     return "no match"
...
...
>>> print(file_type("test.esx"))
zip and formats based on it e.g. pptx/xlsx/docx

Python Versions

Both macOS High Sierra and Mojave (10.14.2) ship with Python 2.7 out of the box. However, Python 2 end of life/depreciation is coming up in 2020. You should have very good reasons to continue writing Python 2 code. Any new projects should be using Python 3.

To check what version of Python is installed run $ python -V or $ python --verson from a terminal:

$ python -V
Python 2.7.10

note “Note” Note the ltter case. For version, the uppercase -V is required. If you use a lowercase -v you will enter a verbose interactive mode

Depending on how you installed Python 3, you may need to check the current installed versions like this:

$ python --version
Python 2.7.10

$ python3 --version
Python 3.7.1

Python Interpreter

Python interpreter is an interactive program that reads and executes Python code.

note “Note” Python interpreter lets you try something out interactively to see how it works without writing up a file. Folks sometimes refer to it with different names: IPython; Read, Evaluate, Print, and Loop (REPL); Python interpreter; Python shell, Interactive Mode. These all refer to the same thing.

It gives you immediate feedback on your code because you don’t need selective print statements in code to see what variables are. >>> object will show what object is:

>>> hello = "Hello, World!"
>>> hello
'Hello, World!'

Try out arithmetic with something like the 2**8. ** is the power operator which will perform exponentiation:

>>> 2**8
256

You can even tab-complete functions and attributes of an object too: object.<TAB>, nice!

>>> str.
str.capitalize(    str.isalpha(       str.ljust(         str.rstrip(
str.casefold(      str.isascii(       str.lower(         str.split(
str.center(        str.isdecimal(     str.lstrip(        str.splitlines(
...

Did you know? help()

Python has a built-in help system intended for interactive use. Type help() for interactive help, or help(object) for help about object. Like help(os), help(magic), help(zipfile), help(str), or help(int).

note “Note” In some cases you may need to import the object first before using help(object), like with os, magic, or zipfile.

Did you know? dir()

If you want to know more about the attributes of an object you can run dir(object). Running dir() without passing in a parameter will output names in the current scope. Use help(dir) to get more information about dir.

Below is an example of using dir on the string module. You can see many attributes and may notice some familiar common ones like .split, .splitlines, .strip, and .lower.

>>> dir(str)
['__add__', '__class__', '__contains__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__getnewargs__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__iter__', '__le__', '__len__', '__lt__', '__mod__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__rmod__', '__rmul__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'capitalize', 'casefold', 'center', 'count', 'encode', 'endswith', 'expandtabs', 'find', 'format', 'format_map', 'index', 'isalnum', 'isalpha', 'isascii', 'isdecimal', 'isdigit', 'isidentifier', 'islower', 'isnumeric', 'isprintable', 'isspace', 'istitle', 'isupper', 'join', 'ljust', 'lower', 'lstrip', 'maketrans', 'partition', 'replace', 'rfind', 'rindex', 'rjust', 'rpartition', 'rsplit', 'rstrip', 'split', 'splitlines', 'startswith', 'strip', 'swapcase', 'title', 'translate', 'upper', 'zfill']

Changing the current directory inside IPython

Import os and then use the .getcwd() and .chdir() inner functions to traverse the filesystem.

>>> import os
>>> os.getcwd()
'/Users/josh/wizardfi.com'
>>> os.chdir("/")
>>> os.getcwd()
'/'

Exiting Interactive Mode

You can exit interactive mode with a keyboard shortcut that sends an end-of-file (EOF) by pressing Ctrl + D. You can also use an interactive shell helper called exit by typing exit():

$ python
...
>>> exit
Use exit() or Ctrl-D (i.e. EOF) to exit
>>> exit()
$

Changelog

  • 2019-09-25: wording
  • 2019-01-19: moved sections, wording, add appendix examples
  • 2019-01-12: clarification/wording, typos, syntax
  • 2019-01-11: original post

Acknowledgments

Thanks to Rowell Dionicio, François Vergés, Tim Smith, Wes Purvis, and Mark Szymanski for reading drafts of this.

↑ Top