Hey everyone,
is it possible to manipulate a VectorSource object in wradlib in a way that its spatially shifted along x and y? I am computing a lot of zonal operations in wradlib with shifting rasters (which get converted to shapes containing the grid centroids, which takes quite long). How could one access the Polygon data and manipulate it to achieve this spatial shift?
Or how is generally possible with gdal to move a shapefile spatially by an x- y-offset? This way I could possibly speed up the process quite a bit by not having to compute this object over and over again.
One solution would be to use ogr2ogr — GDAL documentation. There are two answers on the above SO-question related to that.
I’ve already thought about doing this using OGR Python bindings (there is one answer to that SO-question too), but I think this will be inherently slow too, as you need to iterate over all features separately.
Please let us know, which path you took. A little example use case presented here would be nice too.
Hello @kmuehlbauer,
thank you for this suggestion. It is exactly what I was looking for. I ran into some difficulties when reading these shifted shapefiles with wradlib.io.VectorSource because the name and data format of the index column change. After I fixed this it turned out, that it did not speed up my computations significantly which is why I turned back to the workflow recommended here: A quick start to zonal statistics — wradlib.
Now, instead of shifting the shapefile which contains subbasins for which I wanted to derive the mean precipitation, I shift the radolan dataset (xarray).
Thanks again for the quick help and the never ending revealing of more functionalities of GDAL!
You’re welcome. Glad, that you found a way which works for you. Just asking which way you tried of the suggestions? Maybe there is still a way to improve, but we would need a bit more details on you overall workflow. Can you explain the reason for having to do those shifts? Which distance are you normally shifting? What do you want to achieve by that?
Hello @kmuehlbauer,
I basically shift historical rainfall events to different locations within Germany to investigate the hydrological response (together with my supervisor and your well known colleague mhei****n). This means that the shift along x and y usually is in the range of some hundreds of kilometers. To derive the rainfall series I use the workflow proposed in the wradlib documentation.
There are two approaches: either shift the rainfall event to a desired location, or move the location to the rainfall event. Because the gridvert interpretation of the rainfall event /radolan grid) is quite large and takes some time to compute I was hoping to speed up the process by shifting the location (shapefile with subbasins). For this I used following proposed solution, calling it in a python script with os.system():
ogr2ogr data_shifted.shp data.shp -dialect sqlite -sql "SELECT ShiftCoords(geometry,1,10) FROM data"
In the above example a shapefile is shifted by x=1 and y=10.
I encountered to problems after this shifted shapefile:
The ID field is called “FID” and has Int64 format, whereas wradlib expects the name “trg_index” and Int format.
After reformating the file accordingly (also with ogr2ogr) I still got a lot of warnings regarding the index field of this shifted shapefile
It turned out that my computations did not speed up significantly with this approach, which is why I got back to the worklow which is explained in the wradlib-ZonalStats documentation (awesome job, by the way!).
I don’t think that I will have to do this procedure to often which is why, for now, the slow computation of this process is fine and it can be at least partially be tackled by brute force multiprocessing.
I hope I could answer your questions, thanks for you interest,
Paul
thanks for taking the time to explain the use case. And good to see that my fellow wradlib developer is still connected. Hello Maik . What a neat approach to test river basin response.
We still might have some points of improvement here. Maybe you have already taken one of the following steps.
First, you could create a shapefile which includes all your river basins of interest shifted to a common reference coordinate. They will actally overlay each other, depending on their shape. But for any computations in zonalstats this is no problem (just for visualization this would need to be broken apart).
This reference shapefile could then be exported (wradlib.io.gdal.VectorSource.dump_vector — wradlib) to a more performant shape format, as ESRI Shapefile might not be the best performing here (I’d have to look one up first). Using ogr2ogr you could then move all riverbasins from reference coordinate to the rain location, or move the rain to the common reference point as you are doing now.
Hi @kmuehlbauer,
that is a good approach but I’m afraid, that my shapefiles would be to large. In fact I’m not looking at individual subbasins but at main basins which again contain thousands of subbasins. Overlaying these shapes to cover all positions would require a shapefile with about 25000 overlays, of which each layer has more than a thousand of features. With this size I don’t think it would save a lot of time.
I think I will stick to the approach where the rasters are moved because this is running stable on the cluster already.
Thank you for sharing your ideas!
Paul
. What a neat approach to test river basin response.