Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flamegraph doesn't seem to display user-defined functions #341

Open
jpmedinagl opened this issue Oct 30, 2024 · 7 comments
Open

Flamegraph doesn't seem to display user-defined functions #341

jpmedinagl opened this issue Oct 30, 2024 · 7 comments

Comments

@jpmedinagl
Copy link

I'm working on a project using flamegraph and for some reason I can't get flamegraph to display user-defined functions. These are my configs

[dependencies]
libc = "0.2.155"
faer = "0.19.0"
flamegraph = "=0.5.0"

[profile.dev]
debug = true
opt-level = 0
debug-assertions = true
strip = false

[profile.release]
debug = true
strip = false

I tried running it on my project and I obtain this graph. The code is over 1k lines of code and have many different function calls so I don't know why no functions would show up

This is what I run, I increased the frequency but it doesn't seem to do much.

$ cargo flamegraph --freq 2000 --bin basic

image

I tried creating a random test file to see if I could get a flamegraph non-related to my project. And the flamegraph still doesn't really show the function names. It was suggested on a rust discord to try and use std::hint::black_box to block optimizations and it still doesn't really seem to work.

use std::hint::black_box;

#[no_mangle]
pub fn sum() -> i32 {
    let mut sum = 0;
    for i in 0..1000 {
        sum += std::hint::black_box(1);
    }
    return sum;
}

fn main() {
    println!("Sum: {}", black_box(sum()));
}

image

Is there some setting I'm missing? Right now I want to have a test file that produces a flamegraph with user def functions because I can't seem to do that so unrelated to my project I can't get flamegraph to work

@djc
Copy link
Contributor

djc commented Nov 4, 2024

Sorry, I don't know enough about the underlying functionality in perf (assuming you're profiling on Linux) to be able to answer this question. You might have more luck with samply, maybe (its maintainer certainly knows more about profiling than I do).

@gth828r
Copy link
Contributor

gth828r commented Nov 5, 2024

I was just experiencing the same issue where my flamegraph generated through cargo flamegraph only showed calls that were not part of my codebase. I manually ran the raw binary with perf, used the tools/steps from the original flamegraph repo (https://github.com/brendangregg/FlameGraph) to generate the graph , and this issue went away.

To gather the data, I ran:

perf record -g --call-graph dwarf <repo-path>/target/debug/<bin-name> <my-parameters>

Setting the call graph type to dwarf seemed to be important, and as you can see from the binary path above, I was using a debug build. I am still playing around with things to see what works and what doesn't, but I just wanted to share what I found works so you can play around from a (hopefully) starting point that works and peel back some of the debug pieces that are not necessary.

@djc
Copy link
Contributor

djc commented Nov 5, 2024

@gth828r you can run flamegraph with a pre-existing --perfdata, does that work for you? We also support --cmd which allows you to pass a custom command for invoking perf, so that might be an option as well.

@gth828r
Copy link
Contributor

gth828r commented Nov 6, 2024

@djc This was actually user-error on my part. Others might be having the same self-inflicted issue as me, so I'll write it up for posterity.

In particular, when running the manual steps with perf to generate a flamegraph, I noticed that running perf script takes quite a while, whereas it felt like flamegraph-rs was not taking long enough to be doing the same amount of work.

It turned out that what was happening is that I was sending SIGTERM to terminate my application, and I would get impatient thinking it was still running and send SIGTERM a few more times. That was killing the flamegraph generation process (perf script in particular), but there was still a very basic flamegraph being generated. When running the steps manually, it was quite clear that what was taking long was perf script, but due to the convenience of running cargo flamegraph to roll it all up, it is not obvious what is going on and whether or not something is hanging.

This is pretty minor, but it might make sense to have some type of animated indicator in the terminal to show that work is being done to generate the flamegraph given that the steps can take quite a while (on the order of minutes).

For the record, things ran fine both with a custom command specifying record -g --call-graph dwarf and without the custom command.

@djc
Copy link
Contributor

djc commented Nov 6, 2024

@gth828r thanks for following up. See #270 about discussion on clarifying progress.

@lloydbond
Copy link

Just an FYI, a slow addr2line seems to be the problem as noted here addr2line.

Normally I would profile my program for 5 minutes with cargo flamegraph. Then it would take more than an hour to process and generate a flamegraph.svg. I installed the addr2line from cargo install and now the processing takes less than a minute. I'm currently on Ubuntu 24.04 with binutils 2.42.

@gth828r
Copy link
Contributor

gth828r commented Nov 18, 2024

Just an FYI, a slow addr2line seems to be the problem as noted here addr2line.

Interesting context, thanks for sharing!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants