Converting RDataFrames to awkward array #2892

alig146 · 2023-12-12T03:33:49Z

alig146
Dec 12, 2023

Hi,

I’m trying to convert RDataFrame to awkward array and i seem to get a error with ak.from_rdataframe. Basically i have ROOT and Awkward downloaded via conda and when i run the following:

df3 = ROOT.RDataFrame("CollectionTree", "/global/u2/a/agarabag/pscratch/ditdau_samples/graviton.root")
npy3 = ak.from_rdataframe(df3, columns=("DiTauJetsAuxDyn.ditau_pt", "EventInfoAuxDyn.mcEventWeights"), keep_order=True)

i get the below error:

File "/global/u2/a/agarabag/plotter_v5.py", line 534, in plot_branches
npy3 = ak.from_rdataframe(df3, columns=("DiTauJetsAuxDyn.ditau_pt", "EventInfoAuxDyn.mcEventWeights"), keep_order=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/global/homes/a/agarabag/.conda/envs/ditau/lib/python3.11/site-packages/awkward/_dispatch.py", line 39, in dispatch
gen_or_result = func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/global/homes/a/agarabag/.conda/envs/ditau/lib/python3.11/site-packages/awkward/operations/ak_from_rdataframe.py", line 56, in from_rdataframe
return _impl(rdf, columns, highlevel, behavior, with_name, offsets_type, keep_order)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/global/homes/a/agarabag/.conda/envs/ditau/lib/python3.11/site-packages/awkward/operations/ak_from_rdataframe.py", line 62, in _impl
import awkward._connect.rdataframe.from_rdataframe # noqa: F401
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/global/homes/a/agarabag/.conda/envs/ditau/lib/python3.11/site-packages/ROOT/_facade.py", line 154, in _importhook
return _orig_ihook(name, *args, **kwds)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/global/homes/a/agarabag/.conda/envs/ditau/lib/python3.11/site-packages/awkward/_connect/rdataframe/from_rdataframe.py", line 51, in
cppyy.add_include_path(
File "/global/homes/a/agarabag/.conda/envs/ditau/lib/python3.11/site-packages/cppyy/init.py", line 221, in add_include_path
raise OSError("no such directory: %s" % path)
OSError: no such directory: /global/u2/a/agarabag/.conda/envs/ditau/lib/python3.11/site-packages/awkward/_connect/header-only

This error occurred while calling

ak.from_rdataframe(
RDataFrame-instance
columns = ('DiTauJetsAuxDyn.ditau_pt', 'EventInfoAuxDyn.mcEventWeights')
keep_order = True
)

I wanted to know if anyone knows what could be causing this error? i have tried removing and re downloading awkward but that hasn’t helped. The “header-only” dir indeed does not exist but I’m not sure why. also awkward functions normally its only when calling ak.from_rdataframe that i get this error.

ROOT Version: 6.28.4
awkward version: 2.5.0

ianna · 2023-12-12T06:48:56Z

ianna
Dec 12, 2023
Maintainer

Hi Ali,The header-only is a part of awkward-cpp. Please check if it is installed. Thanks, Yana. Sent from my iPhone

2 replies

alig146 Dec 12, 2023
Author

Hi Yana,

yes, awkward-cpp is installed (version 26). also i tried to downgrade to 2.4 and got the same error and i also jumped to version 2.1.1 and the code is running now.

alig146 Dec 12, 2023
Author

also I wanted to ask a bit unrelated questions. the code works with version 2.1.1 on rather small root files. when i try to run on root files > 1 GB i get "Segmentation fault". i assume this is a memory issue. does this mean my machine does not have enough memory or something more complicated? and also wanted to check that is this a known limit, that is with above some threshold ak.from_rdataframe doesn't work?

ianna · 2023-12-12T16:51:54Z

ianna
Dec 12, 2023
Maintainer

Hi Yana,

yes, awkward-cpp is installed (version 26). also i tried to downgrade to 2.4 and got the same error and i also jumped to version 2.1.1 and the code is running now.

Oh, it looks like a bug. Thanks for reporting it!

I am constructing a path to the header-only directory, but it looks like it is valid only in a developers installation:

awkward/src/awkward/_connect/rdataframe/from_rdataframe.py

Lines 51 to 59 in ae5923e

    
           cppyy.add_include_path( 
        
               os.path.abspath( 
        
                   os.path.join( 
        
                       os.path.dirname(os.path.realpath(__file__)), 
        
                       os.path.pardir, 
        
                       "header-only", 
        
                   ) 
        
               ) 
        
           )

4 replies

agoose77 Dec 12, 2023
Collaborator

This should work in production, even though it won't work with zipfile modules. Perhaps @alig146 could report what the following script yields?

#!/usr/bin/env python3
import awkward._connect.rdataframe.from_rdataframe
import os.path

search_path = os.path.abspath( 
   os.path.join( 
       os.path.dirname(os.path.realpath(awkward._connect.rdataframe.from_rdataframe.__file__)), 
       os.path.pardir, 
       "header-only", 
   ) 
) 
print(search_path)
print(os.listdir(search_path))

agoose77 Dec 12, 2023
Collaborator

Then, which parts of the path that is reported actually exist on your file system?

alig146 Dec 13, 2023
Author

Hi I ran the script. and here are the print results

for awkward version 2.1.1:
first print:
/global/homes/a/agarabag/.conda/envs/ditau/lib/python3.11/site-packages/cppyy_backend/loader.py:90: UserWarning: No precompiled header available (/global/homes/a/agarabag/.conda/envs/ditau/lib/python3.11/site-packages/cppyy_backend/etc not writable); this may impact performance.
warnings.warn('No precompiled header available (%s); this may impact performance.' % msg)
/global/u2/a/agarabag/.conda/envs/ditau/lib/python3.11/site-packages/awkward/_connect/header-only

second print:
['awkward']

for awkward version 2.5.0:
first print:
/global/homes/a/agarabag/.conda/envs/ditau/lib/python3.11/site-packages/cppyy_backend/loader.py:90: UserWarning: No precompiled header available (/global/homes/a/agarabag/.conda/envs/ditau/lib/python3.11/site-packages/cppyy_backend/etc not writable); this may impact performance.
warnings.warn('No precompiled header available (%s); this may impact performance.' % msg)
Traceback (most recent call last):
File "/global/u2/a/agarabag/test.py", line 2, in
import awkward._connect.rdataframe.from_rdataframe
File "/global/homes/a/agarabag/.conda/envs/ditau/lib/python3.11/site-packages/awkward/_connect/rdataframe/from_rdataframe.py", line 51, in
cppyy.add_include_path(
File "/global/homes/a/agarabag/.conda/envs/ditau/lib/python3.11/site-packages/cppyy/init.py", line 221, in add_include_path
raise OSError("no such directory: %s" % path)
OSError: no such directory: /global/u2/a/agarabag/.conda/envs/ditau/lib/python3.11/site-packages/awkward/_connect/header-only

In version 2.1.1 header-only directory exists while in 2.5.0 it does not.

agoose77 Dec 13, 2023
Collaborator

Ah @alig146 I think I'm figuring out what's happened here. Thanks!

ianna · 2023-12-12T16:58:12Z

ianna
Dec 12, 2023
Maintainer

also I wanted to ask a bit unrelated questions. the code works with version 2.1.1 on rather small root files. when i try to run on root files > 1 GB i get "Segmentation fault". i assume this is a memory issue. does this mean my machine does not have enough memory or something more complicated? and also wanted to check that is this a known limit, that is with above some threshold ak.from_rdataframe doesn't work?

I think, an insufficient memory is a valid guess. You can try to monitor it to check it's limit on a file size for your computer. It may depend on what else you are running. The stack trace could provide more detailed information about it.

1 reply

alig146 Dec 13, 2023
Author

when i run over > 1GB file which I think is actually not that large, i just get the Segmentation fault message and the script is terminated there is no other print (i assume this is what you meant by stack trace). also i'm running on perlmutter cluster now so i think the computing resources are fine. if you like to reproduce this i have put the root files, conda env, and script here: https://cernbox.cern.ch/s/BYiAkT0HMXxBCHI

ianna · 2023-12-13T14:44:47Z

ianna
Dec 13, 2023
Maintainer

when i run over > 1GB file which I think is actually not that large, i just get the Segmentation fault message and the script is terminated there is no other print (i assume this is what you meant by stack trace). also i'm running on perlmutter cluster now so i think the computing resources are fine. if you like to reproduce this i have put the root files, conda env, and script here: https://cernbox.cern.ch/s/BYiAkT0HMXxBCHI

Indeed, I've tried to run it on a local copy of your file - it doesn't take long at all - it's not that large:

In [1]: import ROOT

In [2]: import awkward as ak

In [3]: ROOT.ROOT.EnableImplicitMT(16)
Installed ROOT event loop hook.

In [4]: %%timeit
   ...: df3 = ROOT.RDataFrame("CollectionTree", "/Users/yana/Downloads/user.agarabag.34455039._000004.output.root")
   ...: 
   ...: 
The slowest run took 8.06 times longer than the fastest. This could mean that an intermediate result is being cached.
830 µs ± 1 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

retrieving the columns takes a bit longer (the data is copied):

In [7]: %%timeit
   ...: npy3 = ak.from_rdataframe(df3, columns=("DiTauJetsAuxDyn.ditau_pt", "EventInfoAuxDyn.mcEventWeights"), keep_order=True)
   ...: 
   ...: 
Warning in <TClass::Init>: no dictionary for class xAOD::EventFormat_v1 is available
Warning in <TClass::Init>: no dictionary for class ElementLinkBase is available
Warning in <TClass::Init>: no dictionary for class ElementLink<DataVector<xAOD::TruthParticle_v1> > is available
Warning in <TClass::Init>: no dictionary for class ElementLink<DataVector<xAOD::EventInfo_v1> > is available
Warning in <TClass::Init>: no dictionary for class ElementLink<DataVector<xAOD::TrackParticle_v1> > is available
Warning in <TClass::Init>: no dictionary for class xAOD::EventInfo_v1 is available
Warning in <TClass::Init>: no dictionary for class SG::AuxElement is available
Warning in <TClass::Init>: no dictionary for class SG::IAuxElement is available
Warning in <TClass::Init>: no dictionary for class xAOD::EventAuxInfo_v2 is available
Warning in <TClass::Init>: no dictionary for class xAOD::AuxInfoBase is available
Warning in <TClass::Init>: no dictionary for class xAOD::AuxContainerBase is available
Warning in <TClass::Init>: no dictionary for class ElementLink<DataVector<xAOD::TruthVertex_v1> > is available
Warning in <TClass::Init>: no dictionary for class xAOD::TruthParticleAuxContainer_v1 is available
Warning in <TClass::Init>: no dictionary for class xAOD::TruthParticle_v1 is available
Warning in <TClass::Init>: no dictionary for class xAOD::TauJet_v3 is available
Warning in <TClass::Init>: no dictionary for class ElementLink<DataVector<xAOD::Jet_v1> > is available
Warning in <TClass::Init>: no dictionary for class ElementLink<DataVector<xAOD::Vertex_v1> > is available
Warning in <TClass::Init>: no dictionary for class xAOD::DiTauJetAuxContainer_v1 is available
Warning in <TClass::Init>: no dictionary for class xAOD::DiTauJet_v1 is available
Warning in <TClass::Init>: no dictionary for class xAOD::ShallowAuxContainer is available
Warning in <TClass::Init>: no dictionary for class DataLink<SG::IConstAuxStore> is available
Warning in <TClass::Init>: no dictionary for class ElementLink<DataVector<xAOD::TruthEventBase_v1> > is available
Warning in <TClass::Init>: no dictionary for class SG::IAuxStore is available
Warning in <TClass::Init>: no dictionary for class SG::IConstAuxStore is available
Warning in <TClass::Init>: no dictionary for class SG::IAuxStoreIO is available
Warning in <TClass::Init>: no dictionary for class SG::IAuxStoreHolder is available
Warning in <TClass::Init>: no dictionary for class xAOD::IParticle is available
Warning in <TClass::Init>: no dictionary for class DataLinkBase is available
5.21 s ± 90.3 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

and it produces some result:

In [10]: print(ak.flatten(npy3["DiTauJetsAuxDyn.ditau_pt"]))
[0, 2.83e+04, 2.17e+04, 4.03e+04, 0, ..., 0, 2.29e+04, 5.45e+04, 2.6e+05]

I'm using different versions though:

>>> ROOT.__version__
'6.28/04'
>>> ak.__version__
'2.4.5'
>>>

~~I'll try to do~~ the same with awkward 2.5.1.

3 replies

ianna Dec 13, 2023
Maintainer

@alig146 - it is possible that the failing jobs produce a log file somewhere on the cluster. Please, check with your devops. Alternatively, I could suggest to: please, try run it in single thread, run the script with just opening the file, try extracting one column, try to apply a filter before extracting the columns.

alig146 Dec 14, 2023
Author

hi, thank you for testing the script and the suggestions, i will also try to reach out to the devops. regarding your suggestions i tried your suggestions and i only got it to work when extracting just one column as shown below:
npy3 = ak.from_rdataframe(df3, columns=("DiTauJetsAuxDyn.ditau_pt", ), keep_order=True)
then i tried other variables and they also worked (with single column), i also tried multiple columns of other variables and it worked except when i included "EventInfoAuxDyn.mcEventWeights" (even with single column) so i guess it doesn't like this variable for some reason. but its strange that it worked for you. unfortunately i can't try version 2.4.5 because of the header-only error.

alig146 Dec 19, 2023
Author

i also got it to work with newer version. however, when i started running over all my root files i again ran into the Segfault with files > 4 GB. I have put one of these files in Cernbox (user.agarabag.34455039._000003.output.root). If you have time could you please see if you can run this one as well (i have also tried running with 2.5.1).

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Converting RDataFrames to awkward array #2892

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 4 comments 10 replies

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

Converting RDataFrames to awkward array #2892

alig146 Dec 12, 2023

Replies: 4 comments · 10 replies

ianna Dec 12, 2023 Maintainer

alig146 Dec 12, 2023 Author

alig146 Dec 12, 2023 Author

ianna Dec 12, 2023 Maintainer

agoose77 Dec 12, 2023 Collaborator

agoose77 Dec 12, 2023 Collaborator

alig146 Dec 13, 2023 Author

agoose77 Dec 13, 2023 Collaborator

ianna Dec 12, 2023 Maintainer

alig146 Dec 13, 2023 Author

ianna Dec 13, 2023 Maintainer

ianna Dec 13, 2023 Maintainer

alig146 Dec 14, 2023 Author

alig146 Dec 19, 2023 Author

alig146
Dec 12, 2023

Replies: 4 comments 10 replies

ianna
Dec 12, 2023
Maintainer

alig146 Dec 12, 2023
Author

alig146 Dec 12, 2023
Author

ianna
Dec 12, 2023
Maintainer

agoose77 Dec 12, 2023
Collaborator

agoose77 Dec 12, 2023
Collaborator

alig146 Dec 13, 2023
Author

agoose77 Dec 13, 2023
Collaborator

ianna
Dec 12, 2023
Maintainer

alig146 Dec 13, 2023
Author

ianna
Dec 13, 2023
Maintainer

ianna Dec 13, 2023
Maintainer

alig146 Dec 14, 2023
Author

alig146 Dec 19, 2023
Author