OSError: 'WRADLIB_DATA' path 'C:/Users/Admin/wradlib-data' does not exist

Data
,
1

Hi there,

Python version: 3.9.12
wradlib version: 1.16.2

I was trying to run the following code. But it throws me this error.

f = wrl.util.get_wradlib_data_file(file_path[0])
data, metadata = wrl.io.read_radolan_composite(f)

Here file_path[0] is ‘RW-200506.tar.gz’ which is a binary file.

---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
~\AppData\Local\Temp\ipykernel_12220\2806575136.py in <cell line: 1>()
----> 1 f = wrl.util.get_wradlib_data_file(file_path[0])
      2 data, metadata = wrl.io.read_radolan_composite(f)

~\anaconda3\envs\wradlib\lib\site-packages\wradlib\util.py in get_wradlib_data_file(relfile)
    598 
    599 def get_wradlib_data_file(relfile):
--> 600     data_file = os.path.abspath(os.path.join(get_wradlib_data_path(), relfile))
    601     if not os.path.exists(data_file):
    602         raise EnvironmentError(f"WRADLIB_DATA file '{data_file}' does not exist")

~\anaconda3\envs\wradlib\lib\site-packages\wradlib\util.py in get_wradlib_data_path()
    593         raise EnvironmentError("'WRADLIB_DATA' environment variable not set")
    594     if not os.path.isdir(wrl_data_path):
--> 595         raise EnvironmentError(f"'WRADLIB_DATA' path '{wrl_data_path}' does not exist")
    596     return wrl_data_path
    597 

OSError: 'WRADLIB_DATA' path 'C:/Users/Admin/wradlib-data' does not exist
2

@kmuehlbauer Do you have anything to say on this?

Note: Since there is problem with ASCII, I am using binary data files. However, i still encountered the above error while reading the file.

3

First a tar.gz - file is an archive of data files. wradlib expects single data files. So, you would need to iterate over the tar-archive and feed single files to the wradlib-reader.

Second, wrl.util.get_wradlib_datafile is a convenience-function which returns an absolute path to the wanted file inside the wradlib-data repo. We use it for testing and for the wradlib-notebooks. You do not want to use it in normal operation. Just do:

data, metadata = wrl.io.read_radolan_composite(filename)

where filename points to the wanted file.

4

Hello @kmuehlbauer ,

I understood the second point. Here my doubt is with the first point. How can I iterate over the tar archive and feed single files without actually downloading them to the local machine?
Is there any sample code available for achieving this? Here my goal is to read files from 2005 to 2021 (loop through each year and in every year loop through each month and in each month loop through each day and so on…) I found it little difficult to iterate over multiple loops. Is there any documentation available to loop through .tar archive and reading files without downloading them?

5

Hi @SandeepAllampalli,

There might be ways to open a tar-file directly from the external location, but at least I do not know one.

You would at least need to download the tar-archive. You can then iterate over the contents without extracting physically to disk.

With the following code snippet (see RADOLAN Product Showcase — wradlib for more detail) you can achieve that.

import tarfile
fname = "DE1200_RV2210180700.tar.bz2"
# open the tar archive
fp = tarfile.open(fname)
# get the names
names = fp.getnames()
# extract all files into memory stream buffer
buffer = [io.BytesIO(fp.extractfile(name).read()) for name in names]
# name all streams accordingly (!)
for buf, name in zip(buffer, names):
    buf.name = name
# open with radolan reader
ds = wrl.io.open_radolan_mfdataset(buffer)
display(ds)
6

Hi @kmuehlbauer,

Thank you for the reply.