Interpolation of radar and rain gauge data with wradlib

Hello guys! How are you?

Recently, I have been trying to interpolate data from rain gauges and radar here in Brazil to improve accuracy. I have had success applying Gage adjustment methods with, for example, AdjustMultiply, AdjustAdd and AdjustMixed using wradlib.ipol.OrdinaryKriging. However, I’m having problems using wradlib.ipol.ExternalDriftKriging. I would like your help to try to resolve this problem.

Below I provide two examples for comparison. The first works, the second doesn’t.

input:

obs: (85,)
obs_coords: (85, 2)
radar_h: (250000,)
grid_coords: (250000, 2)
radar_v: (250000,)

(1) AdjustAdd using wradlib.ipol.OrdinaryKriging.

%%time
# Additive Error Model with OrdinaryKriging Exponential
# class wradlib.ipol.OrdinaryKriging(src, trg, cov='1.0 Exp(10000.)', nnearest=12)
addadjuster_ok_exp = wrl.adjust.AdjustAdd(obs_coords, 
                                      grid_coords,
                                      cov = '1.0 Exp(10000.)',
                                      ipclass=wrl.ipol.OrdinaryKriging)
addadjusted_ok_exp = addadjuster_ok_exp(obs, radar_h)
#---

(2) AdjustAdd using wradlib.ipol.ExternalDriftKriging.

%%time
addadjuster_kde_exp = wrl.adjust.AdjustAdd(obs_coords, 
                                           grid_coords,
                                           cov = '1.0 Exp(10000.)',
                                           nnearest=12,
                                           src_drift=radar_v, 
                                           trg_drift=grid_coords,
                                           ipclass=wrl.ipol.ExternalDriftKriging)
addadjusted_kde_exp = addadjuster_kde_exp(obs,radar_h)
#---

The following error is displayed:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
File <timed exec>:8

File ~/miniconda3/envs/radar-dev/lib/python3.9/site-packages/wradlib/adjust.py:442, in AdjustAdd.__call__(self, obs, raw, targets, rawatobs, ix)
    440 error = obs[ix] - rawatobs[ix]
    441 # interpolate the error field
--> 442 iperror = ip(error)
    443 # add error field to raw and make sure no negatives occur
    444 return np.where((raw + iperror) < 0.0, 0.0, raw + iperror)

File ~/miniconda3/envs/radar-dev/lib/python3.9/site-packages/wradlib/ipol.py:1362, in ExternalDriftKriging.__call__(self, vals, src_drift, trg_drift)
   1360 src_d = self._make_2d(src_drift)
   1361 trg_d = self._make_2d(trg_drift)
-> 1362 self._check_shape(src_d)
   1364 # re-initialize weights and variances to ensure that these only reflect
   1365 # the results of the current call and not any previous call
   1366 self.weights = []

File ~/miniconda3/envs/radar-dev/lib/python3.9/site-packages/wradlib/ipol.py:119, in IpolBase._check_shape(self, vals)
    109 """
    110 Checks whether the values correspond to the source points
    111 
   (...)
    116 
    117 """
    118 if len(vals) != self.numsources:
--> 119     raise ValueError(
    120         f"Length of value array {len(vals)} does not correspond to number "
    121         f"of source points {self.numsources}"
    122     )
    123 self.valsshape = vals.shape
    124 self.valsndim = vals.ndim

ValueError: Length of value array 250000 does not correspond to number of source points 44

Hi Elton @eltonrobaina ,

welcome to openradar discourse. That question of yours is getting deep into algorithms which have been implemented already over 10 years ago. I’m no expert in these part of wradlib,

We would need to find the exact location in the complete internal workflow to see where this happens. For instance for the Adjust… classes the interpolator (here ExternalDriftKriging) is recreated in case there are invalid points. There might be other issues playing into this.

From the final error message it is clear, that the length of the given data array is not equal to the amount of source points for the final interpolation. I’m assuming some of the source points are dropped but the value array is left intact. That might still be a bug or some regression due to dependency upgrades.

@eltonrobaina would you be able to create a minimized (source-grid, target-grid) version which still breaks and provide the data along with the code? If that doesn’t work, can you provide the data and the code in full size?

Thanks!
Kai

Hi kai @kmuehlbauer

Thanks for the answer.
It’s great to talk to you again (the last time was on the old wradlib-users ).

I prepared a version of the code in Jupyter notebook and I am also making the data available at this link for my Google Drive here

So my idea is simple, as we have a polarimetric radar, I would like to use External Drift Krigin (KDE) to insert vertical precipitation data in addition to horizontal precipitation data and rain gauges to check if it is possible to improve the accuracy.

Thanks!

Hi Elton,

Yes great you found the way to our new community discourse. It’s great to see all coming together.

For your problem, I think I’ve found the issue here. If we look at the docstring wradlib.ipol.ExternalDriftKriging — wradlib, we see that

src_drift : :class:`numpy:numpy.ndarray`
        ndarray of floats, shape (nsrcpoints, )
        values of the external drift at each source point
trg_drift : :class:`numpy:numpy.ndarray`
        ndarray of floats, shape (ntrgpoints, )
        values of the external drift at each target point

src_drift and trg_drift correspond to src (here likely obs) and trg (here probably radar_v). Setting these kwargs with the needed 1D arrays the code runs properly. As I’ve limited knowledge with kriging I can’t say much what values those drift-kwargs expect.

I’ve taken the chance to boost the use of xarray with your notebook. This could be pushed further using apply_ufunc for the Adjustment Part instead of extracting numpy arrays and rewrapping in DataArrays.

HTH,
Kai

1 Like

Hi Kai @kmuehlbauer

Worked perfectly.
I really liked the upgrade you made, it’s always good to learn from experts like you.

Thank you very much for your help again! :grin:

Elton

1 Like

Hello Elton @eltonrobaina,

thanks, glad you find it useful. But, please double (or triple) check that the parameters given with src_drift and trg_drift are the correct ones. I was just assuming (or better guessing) that obs and radar_v derived values are fitting (from their sizes).

Best,
Kai

1 Like

Hi Kai @kmuehlbauer

Positive.
I’m still studying your solution step by step and I’ll be checking and testing new possibilities too (for example using wind speed).
I will also do double and triple checks as per your guidelines.

Thank you again!

1 Like