Release DynamoRIO v9.0.0

[abhinav92003]

From offline discussion with @derekbruening: we want to release a new version of DR soon. It has been a while since the last release, which was in April 2020. We have made several stability fixes since then, particularly some related to memory management options. As some users hit issues with OOM recently (see discussions on https://groups.google.com/g/dynamorio-users), possibly due to sub-optimal defaults, it'll be good to create a new release.

Task list for some more fixes to include in the release is as follows. We can add more to this list if needed.

• Set and fix opcode enums on AArch64 v8.0 for all future releases #5144 (@AssadHashmi)
• Possibly unhandled system calls in Linux kernel 5.10 #5131 (@abhinav92003)
• Call drreg_restore_all prior to clean calls #4128 (@derekbruening)
• add application callstack walking extension #2414 (@derekbruening)
• drcachesim child cache stats should include all lower levels #5190 (@derekbruening)
• samples.dox errors + missing samples (@derekbruening)
• transparency library wording (@derekbruening)
• broken pubs links (@derekbruening)
• App running +rwx code at DR injection time crashes #5198 (@derekbruening)
• Hide map-injection's interpreted return-path blocks from clients? #4958 (@derekbruening)
• Doxygen marks drreg as a "Multi-instrumentation Manager" #2891
• mark -attach as experimental in drrun output and deployment docs (done in PR i#5054: Hide Linux attach until it is fixed #5227)
• Finish AArch64 encoder/decoder #2626
• Use the actual written address range in DC ZVA's memory operand #4400
• Remove superfluous non-operands from LDXR, STXR, and related instructions on AArch64 #4532
• DynamoRIO Incorrectly switch to thumb mode when encountering the MRRC instruction on armv7l #5233
• DrMemory 2.4 and 2.5 Unable to load client library on Windows Server 2008 R2 Enterprise drmemory#2440
• Linux PID attach feature causes application to segfault #5054
• Add AArch64 opcode enum reorderings to changelog
• Add drx_expand() function for VSIB AVX2/AVX-512 scatter/gather. #2985

Nice to have:

• attach injection on Linux #38 for AArch64

[derekbruening]

I may try to implement a libunwind version of #2414 but would not be able to put SEH64 in there in time so it should not gate the release since it would not be an all-platform feature.

[derekbruening]

The other thing is that contributors added externally-triggered attach support (Window: #725 PR #5075; Linux: #38 PR #5019) which is a major feature that we could list for 9.0, but there is no test at all for Linux which makes it hard to claim it works.

@M3m3M4n if you could add an attach test we could then announce attach support for the 9.0 release.

@OrBenPorath if you get a chance to move the Windows attach test off the flaky list that would be great.

[derekbruening]

Xref #1995

Should fix #2891 docs minor issues

[M3m3M4n]

I'm quite tight on time until the end of the year. Will try to add the tests in my free time.
Also I think #5054 need to be fixed first, I haven't read too much into DR core aside from attaching part, so need more time to iron things out.

[derekbruening]

@M3m3M4n the attach feature is causing confusion with users who try to use it and complain that it's broken: https://groups.google.com/g/dynamorio-users/c/0n_cgKDPM_M/m/nRLdlKQQAwAJ?utm_medium=email&utm_source=footer Would be great if you could help reply to questions like this, and help clearly mark it as experimental in the docs since it seems to not work everywhere

[ManojKumarChauhan]

Hi @M3m3M4n, per my understanding there are 2 ways to use wrrap for any client application.

Suppose I have a client C application testclient which contains some functions like malloc, function1, function2
To wrap malloc, I can either do it using attach option or by giving the app name.

1. Attach option
   Run testclient from terminal one (pid-14948). From another terminal, use following command
   bin64/drrun -attach 14948 -c ./api/bin/libwrap.so

It does nothing and got stack while testclient is still running

2. Using app name
   bin64/drrun -client ./api/bin/libwrap.so 0 -- testclient
   It starts to run testclient but I never get the access of testclient's functions (malloc, function1...)

As per https://dynamorio.org/page_deploy.html, following command is used to wrap malloc function
bin32/drrun -c ./api/bin/libwrap.so -- ls
But I never get the access of malloc function of ls.
module_load_event is called only for libwrap.so and its dependencies and their malloc function.
mod->names.module_name prints libwrap.so and its dependencies. It never find ls binary's module.

I compiled Dynamorio from source code using
git clone https://github.com/DynamoRIO/dynamorio
mkdir build_dr
cd build_dr/
cmake ../dynamorio/
make -j

[M3m3M4n]

Hi @ManojKumarChauhan, as stated in issue #5054. There is an unknown bug somewhere in the way DR load after injection. Could you build DR with -DDEBUG=ON and see if it works?

[M3m3M4n]

@derekbruening I'm not very familiar with google mail group. Can you tell them to try to build with DDEBUG option enabled as in #5054 for me? Thank you.

And I almost done with my company project. If the issue hasn't been fix by then I will see to it.

[ManojKumarChauhan]

Hi @M3m3M4n
Sure, I will try with debug option and update you.

[ManojKumarChauhan]

Same issue while compiling Dynamorio in debug mode. When I close client application, drrun gives following error
ERROR: unable to inject: exec of |(null)| failed
So attach to process functionality is not working.

For second case where we give the application name (bin64/drrun -client ./api/bin/libwrap.so 0 -- testclient), it's working only when the function name is malloc because it finds that function from any of the dependent library of libwrap. But if I give any other function which is implemented in testclient then it's unable to find that function because module_load_event doesn't find any module of testclient.

[derekbruening]

2. Using app name
   bin64/drrun -client ./api/bin/libwrap.so 0 -- testclient
   It starts to run testclient but I never get the access of testclient's functions (malloc, function1...)

As per https://dynamorio.org/page_deploy.html, following command is used to wrap malloc function bin32/drrun -c ./api/bin/libwrap.so -- ls But I never get the access of malloc function of ls. module_load_event is called only for libwrap.so and its dependencies and their malloc function. mod->names.module_name prints libwrap.so and its dependencies. It never find ls binary's module.

As the docs say, module_name is NULL for a non-.so: so you would expect it to be NULL for ls. Again as the docs say, use dr_module_preferred_name().

I compiled Dynamorio from source code using git clone https://github.com/DynamoRIO/dynamorio mkdir build_dr cd build_dr/ cmake ../dynamorio/ make -j

Looks like this works fine, with module_load_event printing everything it sees:

$ cmake -GNinja ../src
$ ninja
$ bin64/drrun -client ./api/bin/libwrap.so 0 -- ls
Client wrap is running
module_load_event: libwrap.so
module_load_event: ls
module_load_event: ld-linux-x86-64.so.2
module_load_event: libdynamorio.so
module_load_event: linux-vdso.so.1
module_load_event: libc.so.6
<wrapped libc.so.6!malloc @0x00007f60721996c0
module_load_event: libpthread.so.0
module_load_event: libdl.so.2
module_load_event: libpcre2-8.so.0
module_load_event: libselinux.so.1
api	     cmake			 CMakeFiles	      configure_temp.h	       docs		   event_strings.h  libutil
bin64	     CMakeCache.txt		 cmake_install.cmake  core		       drcachesim.drrun64  ext		    logs
build.ninja  CMakeDoxyfile.in		 configure_defines.h  CPackConfig.cmake        drcov.drrun64	   include	    tools
clients      CMakeDoxygenDefaults.cmake  configure.h	      CPackSourceConfig.cmake  drcpusim.drrun64    lib64
<Largest malloc request: 32816>
<OOM simulations: 0>

But, please separate further questions you have about drwrap from this issue: please direct them to the users list https://groups.google.com/forum/#!forum/DynamoRIO-Users for help and discussion and if that discussion concludes there is an underlying bug it would be filed separately (and possibly linked here if it affects the release).

[derekbruening]

@derekbruening I'm not very familiar with google mail group. Can you tell them to try to build with DDEBUG option enabled as in #5054 for me? Thank you.

I replied saying it's still experimental. I tried debug build myself and it crashes there too on my machine.

And I almost done with my company project. If the issue hasn't been fix by then I will see to it.

I put in PR #5227 which hides the feature on Linux since it is not ready yet. As part of the fix for #5054 that hiding can be removed.

[derekbruening]

I went and figured out and fixed #5054 along with some other attach bugs and reversed the attach feature hiding. However, there are missing generated code implementations for AArchXX, so this is an x86-only feature until someone implements those.

[derekbruening]

I think the only thing left is PR #5235 (unless someone wants to try to get AArch64 attach working)

[AssadHashmi]

I think the only thing left is PR #5235 (unless someone wants to try to get AArch64 attach working)

Hoping to complete #5235 in the next day or so. AArch64 attach is work which we can't commit to at this time.