Error in phase processing

Corrections/Data Quality
1

I am using a C band radar with radar fields [‘DBZ’, ‘PHIDP’, ‘RHOHV’, ‘VEL’, ‘WIDTH’, ‘ZDR’]
for phase processing i was using this code
phidp, kdp = pyart.correct.phase_proc_lp(radar, 0.0,refl_field=‘DBZ’,phidp_field=‘PHIDP’,rhv_field=‘RHOHV’,ncp_field=‘RHOHV’ ,LP_solver = “cvxopt”,debug=True)
error :
Cell In[4], line 6
1 # the next line force only the first sweep to be processed, this
2 # significantly speeds up the calculation but should be commented out
3 # in production so that the entire volume is processed
4 radar.sweep_start_ray_index[‘data’] = np.array([0])
----> 6 phidp, kdp = pyart.correct.phase_proc_lp(radar, 0.0,refl_field=‘DBZ’,phidp_field=‘PHIDP’,rhv_field=‘RHOHV’,ncp_field=‘RHOHV’ ,LP_solver = “cvxopt”,debug=True)
8 radar.add_field(‘proc_dp_phase_shift’, phidp)
9 radar.add_field(‘recalculated_diff_phase’, kdp)

File ~\anaconda3\Lib\site-packages\pyart\correct\phase_proc.py:1196, in phase_proc_lp(radar, offset, debug, self_const, low_z, high_z, min_phidp, min_ncp, min_rhv, fzl, sys_phase, overide_sys_phase, nowrap, really_verbose, LP_solver, refl_field, ncp_field, rhv_field, phidp_field, kdp_field, unf_field, window_len, proc, coef)
1192 mysoln = LP_solver_pyglpk(
1193 A_Matrix, B_vectors, nw, really_verbose=really_verbose
1194 )
1195 elif LP_solver == “cvxopt”:
→ 1196 mysoln = LP_solver_cvxopt(A_Matrix, B_vectors, nw)
1197 elif LP_solver == “cylp”:
1198 mysoln = LP_solver_cylp(
1199 A_Matrix, B_vectors, nw, really_verbose=really_verbose
1200 )

File ~\anaconda3\Lib\site-packages\pyart\correct\phase_proc.py:720, in LP_solver_cvxopt(A_Matrix, B_vectors, weights, solver)
718 this_soln = np.zeros(n_gates)
719 for i in range(n_gates):
→ 720 this_soln[i] = sol[“x”][i + n_gates]
722 # apply smoothing filter and record in output array
723 mysoln[raynum, :] = smooth_and_trim(this_soln, window_len=5, window=“sg_smooth”)

TypeError: ‘NoneType’ object is not subscriptable

if anyone has insights on how to solve this I would greatly appreciate your guidance

2

Could you run

conda list

and post the output so we can have an idea of what versions you have installed? Also, which data file is this?

3

conda list
conda list617×630 97.9 KB

data file is from Thumba DWR data which is processed by me . i have attched a sample file below.

4

Which version of Py-ART are you running? The screenshot cuts off in the b portion of the packages.

I tested this on the latest version Py-ART, and it ran successfully:

import pyart

# Read in the radar data
radar = pyart.io.read("RCTLS_13MAY2018_135449_L2B_STD.nc")

# Apply the phase processing
phidp, kdp = pyart.correct.phase_proc_lp(radar,
                                         0.0,
                                         refl_field="DBZ",
                                         phidp_field="PHIDP",
                                         rhv_field="RHOHV",
                                         ncp_field="RHOHV",
                                         LP_solver = "cvxopt",
                                         debug=True)

Unfolding
/Users/mgrover/git_repos/pyart/pyart/correct/phase_proc.py:177: RuntimeWarning: invalid value encountered in divide
  return abs(signal) / _noise
Exec time:  2.717604875564575
Doing  0
GLPK Simplex Optimizer 5.0
Doing  1
6526 rows, 2612 columns, 13038 non-zeros
      0: obj =   0.000000000e+00 inf =   1.071e+04 (2606)
   6322: obj =   1.739944370e+05 inf =   7.283e-14 (0) 60
*  6500: obj =   1.722717219e+05 inf =   9.104e-15 (0) 2
OPTIMAL LP SOLUTION FOUND
GLPK Simplex Optimizer 5.0
5021 rows, 2010 columns, 10028 non-zeros
      0: obj =   0.000000000e+00 inf =   1.472e+04 (2004)
Perturbing LP to avoid stalling [3618]...
   4415: obj =   3.303218515e+03 inf =   1.965e-06 (0) 42
Removing LP perturbation [5042]...
*  5042: obj =   1.192051355e+03 inf =   1.315e-13 (0) 6
OPTIMAL LP SOLUTION FOUND
Doing  2
GLPK Simplex Optimizer 5.0
3251 rows, 1302 columns, 6488 non-zeros
Doing       0: obj =   0.000000000e+00 inf =   9.807e+03 (1296)
Perturbing LP to avoid stalling [1663]...
   2239: obj =   1.739060931e+03 inf =   2.646e-09 (0) 20
Removing LP perturbation [2684]...
*  2684: obj =   3.778228050e+02 inf =   1.972e-13 (0) 4
OPTIMAL LP SOLUTION FOUND
 3
GLPK Simplex Optimizer 5.0
2336 rows, 936 columns, 4658 non-zeros
Doing  4
      0: obj =   0.000000000e+00 inf =   6.854e+03 (930)
   1833: obj =   4.101107728e+02 inf =   7.105e-14 (0) 17
*  2050: obj =   2.530848447e+02 inf =   8.384e-13 (0) 2
OPTIMAL LP SOLUTION FOUND
GLPK Simplex Optimizer 5.0
1806 rows, 724 columns, 3598 non-zeros
      0: obj =   0.000000000e+00 inf =   5.127e+03 (718)
   1166: obj =   3.333312791e+02 inf =   0.000e+00 (0) 10
*  1341: obj =   1.760857907e+02 inf =   2.194e-13 (0) 1
OPTIMAL LP SOLUTION FOUND
Doing  5
GLPK Simplex Optimizer 5.0
1061 rows, 426 columns, 2108 non-zeros
Doing  6
      0: obj =   0.000000000e+00 inf =   2.693e+03 (420)
    740: obj =   1.975207042e+02 inf =   1.315e-13 (0) 6
*   817: obj =   1.008230690e+02 inf =   4.619e-14 (0) 1
OPTIMAL LP SOLUTION FOUND
GLPK Simplex Optimizer 5.0
831 rows, 334 columns, 1648 non-zeros
Doing  7
      0: obj =   0.000000000e+00 inf =   1.950e+03 (328)
    512: obj =   1.378641183e+02 inf =   5.329e-15 (0) 4
*   604: obj =   6.297361663e+01 inf =   5.151e-14 (0) 1
OPTIMAL LP SOLUTION FOUND
GLPK Simplex Optimizer 5.0
626 rows, 252 columns, 1238 non-zeros
Doing  8
      0: obj =   0.000000000e+00 inf =   1.351e+03 (246)
    377: obj =   1.071771379e+02 inf =   1.427e-14 (0) 3
*   429: obj =   5.187844123e+01 inf =   5.063e-14 (0)
OPTIMAL LP SOLUTION FOUND
GLPK Simplex Optimizer 5.0
501 rows, 202 columns, 988 non-zeros
Doing  9
      0: obj =   0.000000000e+00 inf =   9.900e+02 (196)
    309: obj =   8.190416547e+01 inf =   2.320e-14 (0) 2
*   355: obj =   4.393185547e+01 inf =   1.954e-14 (0) 1
OPTIMAL LP SOLUTION FOUND
Doing  10
GLPK Simplex Optimizer 5.0
421 rows, 170 columns, 828 non-zeros
      0: obj =   0.000000000e+00 inf =   7.649e+02 (164)
    234: obj =   7.688575510e+01 inf =   3.553e-15 (0) 1
*   274: obj =   4.094557344e+01 inf =   3.375e-14 (0) 1
OPTIMAL LP SOLUTION FOUND
GLPK Simplex Optimizer 5.0
361 rows, 146 columns, 708 non-zeros
      0: obj =   0.000000000e+00 inf =   6.059e+02 (140)
    203: obj =   7.842866003e+01 inf =   3.553e-15 (0) 1
*   243: obj =   3.780782667e+01 inf =   0.000e+00 (0) 1
OPTIMAL LP SOLUTION FOUND
1 Like
5

@Jagdish_jena - this line is the culprit. What is the reasoning for this line?

radar.sweep_start_ray_index[‘data’] = np.array([0])

This will corrupt the radar object, leading to mismatched array lengths.

1 Like
6

@Jagdish_jena - did this solve the issue?

1 Like
7

Actually the file i have sent you it is working . This file was taken from thumba radar and has clutter and some time and sweep parametre is missing .so i have processed it and convert it into a nc file . That processed file is not working using that code. I am sending you a sample . please check .

8

There is another problem i am facing i.e after processing through that code results value are coming very absurd . just check following figures

  1. PHIDP (raw data )
    PHIDP_RAW
    PHIDP_RAW956×519 39.7 KB

    2.calculated PHIDP
    processed PHIDP
    processed PHIDP967×498 35.3 KB

    3.UNFOLDED PHIDP - zero
    4.KDP
    calculated KDP
    calculated KDP953×512 40.7 KB

i dont know what is the problem . it is the problem with data or methods . because i have tried everything pyart , wradlib, titan didn’t get any satisfied result.

9

@mgrover1 have u checked above figures ?

10

I have not had a chance yet - plan to take a look this week.

1 Like
11

I suspect this is related to the data quality - I tried with multiple methods and found the same results.

1 Like