includes("**/xmake.lua") 卡死 #5052
Comments
Title: includes("**/xmake.lua") stuck Xmake versionxmake v2.8.9+dev.50dbca648 Operating system version and architectureUbuntu 2204 Describe the problemOnce you use includes("/xmake.lua"), it will cause stuck, but there is no problem if you use it one by one.
Because another computer also uses this xmake commit-id version, the problem with the xmake version is also ruled out. If it can be debugged, it should be easy to find the problem, but the emmylua plug-in also runs in the xmake process, and it gets stuck before reaching the debugging stage: Expected resultsI want to know the reason for the stuck, and I suspect it has something to do with the local project, but the project is too big to construct a minimal demo. Or are there any other debugging methods? Project configurationNone yet Additional information and error logsnone. |
|
估计是 includes 匹配的里面，又匹配到了带有 xmake/xmake/core/base/interpreter.lua Line 1688 in 57a6b50 |
It is estimated that in the includes match, xmake.lua with xmake/xmake/core/base/interpreter.lua Line 1688 in 57a6b50 |
ok，我打日志看看 |
OK, I'll check the logs |
|
解决了么 |
Is it solved? |
还没解决，卡在这一行： 日志如下： |
|
找到了一种调试的方法，不用一直打日志，我先调一下 |
到这为止就没法继续往下调试了： rootdir “.”
|
|
如果到此为止参数都是正确的，那我就重装 xmake 吧，如果参数有问题，那我就接着调试看一下哪一行出的问题 |
Not solved yet, stuck on this line: The log is as follows: |
I found a way to debug without logging all the time. I'll adjust it first. |
At this point, there is no way to continue debugging: rootdir "."
|
If the parameters are correct so far, then I will reinstall xmake. If there is a problem with the parameters, then I will continue debugging to see which line is the problem. |
|
那你这个估计是走 你可以设置 callback 看下具体遍历到了哪里。。 local file = os.match("**/xmake.lua", true, function (filepath, isdir)
print(filepath)
return true
end)也可能是你 .git 目录下子文件太多 |
还是直接卡住了没打印什么东西，这个项目确实很大，但是 find . -name xmake.lua 很快就返回了。 |
|
那你只能继续调下 c 里面的遍历了。。 xmake/core/src/xmake/os/find.c Line 36 in f66ae55 拉下 xmake 源码，然后上面地方 通过 printf 或者 tb_trace_i 去 打印下 内部执行，看看遍历到哪了 ./configure
make
source ~/scripts/srcenv.profile
xmake build再不行，就只能调下 tbox 里面的遍历接口了。。https://github.com/tboox/tbox/blob/c6b0a56076941b8263e162c7fe7b0870ea44e09c/src/tbox/platform/posix/directory.c#L270 |
好的，我再试试，另外这个能通过 includes("/xmake.lua|.git//xmake.lua") 来跳过 .git 下面的遍历吗 |
xmake 4w 个文件的遍历能抗住吗
|
|
|
没压测过，但也许会很慢，通常我不会 includes 里面这么干。。 |
|
你可以自己 |
|
卡住了至少得一分钟还没返回。。。 这能优化一下性能吗，至少 find 是立马返回了。虽然这个项目是比较大一点，但是 git 项目 5000 个文件以上还是很普遍的 |
但是这是官方文档的写法，对于大型项目多目录里面都有 xmake.lua 的话挺好用的。 |
find 是纯c的，xmake 里每个遍历，内部都要调度 lua 接口交互，还得每次做 lua pattern 模式匹配，多少会影响性能，但避免不了，而且遍历结果还得塞进 lua table 受限于 lua 的资源管理 |
|
但应该也不会太慢，顶多比 find 慢一点点而已，这边 llvm 12w 的文件，find 遍历 2s 。 xmake 里面遍历，算上 lua 模式匹配 + print 回显一些耗时，总共也就 10s 。。你这才 4w，按理顶多 4s 的时间。。。卡住肯定是其他问题，这个就只能你自己调 llvm ruki$ time find ./ -type f | wc -l
126189
real 0m1.946s
user 0m0.145s
sys 0m1.629sllvm ruki$ time xmake l os.files "**" > /tmp/a
real 0m10.200s
user 0m4.564s
sys 0m5.304s |
我感觉也是哪里卡死了，打日志调试一下下面的 C 代码看看 |
Then you can only continue to adjust the traversal in c. . xmake/core/src/xmake/os/find.c Line 36 in f66ae55 Pull down the xmake source code, and then use printf or tb_trace_i to print the internal execution to see where it has been traversed. ./configure
make
source ~/scripts/srcenv.profile
xmake buildIf that doesn't work, you can only adjust the traversal interface in tbox. . https://github.com/tboox/tbox/blob/c6b0a56076941b8263e162c7fe7b0870ea44e09c/src/tbox/platform/posix/directory.c#L270 |
It has been stuck for at least a minute and has not returned. . . Can this optimize performance? At least find returns immediately. Although this project is relatively large, it is still common for git projects to have more than 5,000 files. |
But this is how it is written in official documents. It is very useful for large projects if there are xmake.lua in multiple directories. |
find is pure C. Every traversal in xmake requires internal scheduling of Lua interface interaction, and Lua pattern matching has to be done every time. It will affect the performance to some extent, but it cannot be avoided. Moreover, the traversal results have to be stuffed into Lua table, which is limited. Resource management in lua |
But it shouldn’t be too slow, at most a little slower than find. For the llvm 12w file here, find traverses 2s. The traversal in xmake, including lua pattern matching + print echo, takes some time, and the total time is only 10s. . You are only 4w, so it should take you 4s at most. . . If it gets stuck, it must be due to other problems. You can only adjust this yourself. llvm ruki$ time find ./ -type f | wc -l
126189
real 0m1.946s
user 0m0.145s
sys 0m1.629sllvm ruki$ time xmake l os.files "**" > /tmp/a
real 0m10.200s
user 0m4.564s
sys 0m5.304s |
I feel like it's stuck somewhere. Let's log and debug the C code below. |
|
而且按你目前的 xmake.lua 结构，可以改成 |
And according to your current xmake.lua structure, you can change it to |
挨个试了一下，删掉 blade-bin 文件夹就可以了，暂时这么写绕过去，但是还是不懂为什么挂在 C++ 函数里： includes("**/xmake.lua|blade-bin/**/xmake.lua") |
I tried them one by one and deleted the blade-bin folder. I wrote this to bypass it for the time being, but I still don’t understand why it hangs in the C++ function: includes("**/xmake.lua|blade-bin/**/xmake.lua") |
|
这个只有你这调下 c 层代码才行 |
This can only be done if you adjust the c-layer code |
|
我猜你的这个
文件夹里面有到上层的软链。
|
I guess this is yours
There is a soft link to the upper level inside the folder.
|
确实，bazel 和 blade 都会生成软链。 |
Indeed, both bazel and blade generate soft links. |
|
话说这个有软链（即使软链里面没有 xmake.lua 文件）会卡死的问题算 bug 吗 |
By the way, is the problem of getting stuck if there is a soft link (even if there is no xmake.lua file in the soft link) considered a bug? |
估计是软链到上层导致递归死循环遍历了，这种目前也区分不了 |
It is estimated that the soft link to the upper layer caused a recursive infinite loop traversal. |
好的，我看看 bazel 都生成了些啥玩意 |
Okay, let me see what bazel has generated. |
不过这里检测到循环抛出来一个 warning 或者异常是不是更友好一点，之前我个人的体验是直接卡死了，没有调试经验的话完全不知道发生了什么 |
However, if a warning or exception is detected here and the loop is thrown, would it be more friendly? My personal experience before was that it was directly stuck. If I have no debugging experience, I have no idea what happened. |
检测这种就需要通过 map 维护已经遍历过的路径状态，影响遍历性能，尤其是遍历大规模文件时候，还影响内存。。而且这种case 出现概率太低。。为了检测这种，去做更复杂的处理，没必要 |
Detecting this requires maintaining the traversed path status through map, which affects traversal performance, especially when traversing large-scale files, and also affects memory. . And the probability of this case happening is too low. . In order to detect this, it is not necessary to do more complex processing |
okok，那先维持现状吧 |
Okok, let’s maintain the status quo for now. |
Xmake 版本
xmake v2.8.9+dev.50dbca648
操作系统版本和架构
Ubuntu 2204
描述问题
一旦使用 includes("/xmake.lua") 就会导致卡死，但是逐个逐个使用就没有问题。
我一开始怀疑是有 xmake.lua 文件也用了 includes("/xmake.lua") 导致循环解析，所以将当前文件夹下所有的 xmake.lua 挨个添加排查：
因为另一台电脑也是用的这个 xmake commit-id 的版本，所以也排除是 xmake 版本的问题。如果可以调试的话应该是很容易发现问题的，但是 emmylua 插件也是在 xmake 进程内运行的，还没走到调试阶段就卡住了：
期待的结果
想知道卡死的原因，怀疑和本地项目有关系，但是项目太大不方便构造一个最小 demo。或者有没有其他的调试方法。
工程配置
暂无
附加信息和错误日志
无。
The text was updated successfully, but these errors were encountered: