Removing data smoothed across azimuths

[hvagasky]

Hi

I have run into a few cases whereradar data is clearly smoothed/extrapolated across several azimuths. An example is shown below. Is there a good/efficient way to identify and remove this data besides hand analysis and removal?

.

Thanks
Hannah

[jorahu]

Hey

Assuming you have polar data you can find inspiration here.

I hope I interpreted the problem correctly, but I tried to replicate something similar.
Inital data had 0.5deg azimuth resolution:

Replicated test data:

sweep[300:400] = sweep[300]
sweep[600:650] = sweep[600]

How it looks:

Numpy unique (next step) did not like nan values. If you do not have nan values ignore this step.
sweep = np.where(np.isnan(sweep), -32000, sweep) # The value is arbitrary

The following is taken from the StackOverflow answer:

unq, count = np.unique(sweep, axis=0, return_counts=True)
repeated_groups = unq[count > 1]

for repeated_group in repeated_groups:
    repeated_idx = np.argwhere(np.all(sweep == repeated_group, axis=1))
    repeated_idx_raveled = repeated_idx.ravel()
    print(repeated_idx_raveled) # Find the repeating rows
    # If you'd like to automate this, here should be if-else or try-except to check if any section is repeating or not
    first_idx = repeated_idx_raveled[0] # First index on the repeated section
    last_idx  = repeated_idx_raveled[-1] + 1 # Last index of the repeated section, notice the +1 because of the way it counts the rows
    sweep[first_idx:last_idx] = np.nan # "Delete" the repeated sections

Output:

Switch back to nan-s (if necessary)
sweep = np.where(sweep==-32000,np.nan, sweep)

Final sweep:

Sorry, I don’t know how to do it directly with xarray structures.
Hope this helps.