Cool project. I think false positives are more important than the dead code it misses at this stage. For example, it marks all Django settings as unused variables, probably because settings are typically not imported directly in Django projects. For example:
from django.conf import settings
...
if settings.DEBUG:
...
Currently unref thinks that DEBUG is not used.
Similarly, the `application` variable in asgi.py (or wsgi.py) is passed as an argument to the ASGI/WSGI server executable, but that is also marked as unused.
I wish they made Django a little less “magic”. Virtually every harness has to create workarounds specifically for Django, and simple things like settings probably don’t need to break standard coding practices.
I got a lot of false positives in a Django project too, but trying it on some non-framework tools I've built worked pretty well. Will keep an eye on this, thanks.
I've been trying to do something like this internally too. Also started with Python, then moved onto ruff. My goal is a bit broader though. I want to use the dependency graph to detect changed parts of the project to selectively trigger image builds and run tests.
Results are still poor, specially on the tests side. So far the main learning has been that a tidy codebase (dep graph) is essential for it to work properly.
I'm surprised I didn't find more open source tooling tackling this effectively.
Did you compare what you’ve built to some of the internals of nuitka? nuitka stops dead code search at the module level, but their exclusion/inclusion flags/rules might steer you towards fewer false positives for common packages.
What is the definition of dead code that Unref uses? I distinguish between apps and libraries. In apps I can see how any code not callable from the entrypoint is dead, and for libraries I guess a definition is any code that is not callable from a public api?
Would be nice to see the definition used placed in the docs/github readme.
Yeah exactly, for apps it's any code that is not reachable from the app entry point, and for libraries is any code not reachable through the public API.
I've updated the README and added a FAQ section in the docs to explain this, thanks for the question!
i see lots of projects for python that spend a ton of duplicated effort trying to make an imperfect tool for something that in other languages can be done reliably with a single keystroke. python resists tooling
The initial version was in Python which was pretty fast as well, decided to try Rust both for speed and to reuse more constructs from ruff/ty for better compatibility with them.
Similarly, the `application` variable in asgi.py (or wsgi.py) is passed as an argument to the ASGI/WSGI server executable, but that is also marked as unused.
I'll make a PR to add some basic Django support which should address both of your points.
Results are still poor, specially on the tests side. So far the main learning has been that a tidy codebase (dep graph) is essential for it to work properly.
I'm surprised I didn't find more open source tooling tackling this effectively.
Would be nice to see the definition used placed in the docs/github readme.
I've updated the README and added a FAQ section in the docs to explain this, thanks for the question!
Here's the PR - https://github.com/Apakottur/unref/pull/121
The initial version was in Python which was pretty fast as well, decided to try Rust both for speed and to reuse more constructs from ruff/ty for better compatibility with them.
Feel free to open a GitHub issue with a few of the false positives: https://github.com/Apakottur/unref/issues
Feel free to open an issue on github if you find any false positives/negatives.
The benchmark script is in the repo so you can run it yourself, the results are from my potato computer, will probably be even faster on modern PCs.
I've updated the README to be more clear about that, thanks for pointing that out!