How long should the pyart python import take?

Python
1

I have been using pyart in python for a little bit. I was wondering: when I import pyart in python, is it normal for it to take 40-60 seconds to import? If so, I will stop worrying about it. If not, I will keep trying to find the issue. It is quite variable how long it takes: sometimes it takes just a couple seconds (it must be cached somehow), but usually it takes nearly a minute.

2

Hi @dmbrown1 , I would say usually not. Or at least I’ve never had it take that long. I will have to have a think on what could maybe cause that. Can also see if others respond and have ideas.

3

Hi @dmbrown1. Are you running from a local install or remotely? I’ve previously had a problem with the import taking a long time when PyART was installed on a compute cluster. It turned out the disk storage I’d used for the install wasn’t optimised for reading lots of small files and this would cause a long delay on import. I moved the python install to some different disk space on the cluster which was optimised for smaller files and things improved massively after that.

4

Hi @dmbrown1,

welcome to openradar discourse.

To better debug this you can check the import times with:

$ python -X importtime -c "import pyart"

A bit more details on your overall setup (OS, Python installation and environment, pyart version etc.) would be helpful. Working over network (as @DavidDufton suggested) can be one source of long import times, other sources are non-lazy import of dependencies.

HTH,
Kai

5

I am primarily using Jupyterlab IDE, and when initialising the session, the first import usually takes longer, around 30 - 60 seconds. Importing packages a second time is usually “fast” (a couple of seconds). This is not package-specific.

6

Hello,

Thanks everyone for their suggestions. I am running it remotely on a cluster. We have a home directory, limited to 20GB, and a storage directory with a lot more space. I had moved all of my conda environments to the storage directory because they were taking too much space on the home directory.

When I moved the pyart conda environment back to my home directory, the performance is improved substantially - 4 seconds to load instead of 45-60 seconds. That seems to be the answer. I will contact the administrators of the cluster to see if there are other solutions, but, for now, leaving it in the home directory will work. I’ll just have to clear out some space for it.

3 Likes