Waaat :D. This is so nice. I used to use python-ls[0] for similar, but something about it that I can't recall broke for me and it's no longer maintained. Adding to my debugging arsenal which primarily consists of snoop[1] and pdbpp. Only thing I'd like for wat now is maybe an ipy widget for even easier object exploration in Jupyter.
I'm also really appreciating the base64 exec hack. All my years in Python I never thought of or came across it until now. I'll totally be using it for some things :).
Ah this looks fun! I use "dir" all the time with python, and find it more useful than official documentation in some cases where documentation is not great.
Surprised there isn't more innovation and new tools like this around python's interactive shell given it's one of the real strong points the language has.
And if you use IPython, you can use the ? shorthand, too:
In [1]: int?
Init signature: int(self, /, *args, **kwargs)
Docstring:
int([x]) -> integer
int(x, base=10) -> integer
Convert a number or string to an integer, or return 0 if no arguments
are given. If x is a number, return x.__int__(). For floating point
numbers, this truncates towards zero.
[...]
2. Put an import pdb;
pdb.set_trace() whereever I need to fill out next
3. Once I hit the breakpoint, type 'interact'
4. Start coding in the debugger.
4a. For small functions write in a scratch pad, test against another python console, then paste into the current debugger.
4b. For network calls, save the inputs and outputs. Toss a decorator on some functions to log the inputs and outputs to files, and cool we already have some proto test data.
4c. For documentation, call help() on the variable or method or module.
Given the cool nature of this project, I'm surprised they don't offer simply "import wat" with identical usage syntax. Thus inviting curious users to wat/wat in order to discover the trick...
"import wat" would be great, but Python has some restrictions about modules not being callable. That's why I ended up with longer `from wat import wat`. Not sure but maybe this would be more convenient `import wat; wat.wat / object`
Check out the module “q”! It’s callable, its author talked about how great it would be to have a module level __call__ because the way it was made callable is super wonky.
According to the official python documentation [0] of sys.modules:
> This is a dictionary that maps module names to modules which have already been loaded. This can be manipulated to force reloading of modules and other tricks.
Thus, the modules dictionary is being used as intended, and it has the desired effect. Nothing wonky about that.
Of course, a __call__ method would be better because you could still keep the other functions inside the module. But for a single-function import like "wat" it seems quite natural and sane.
Thank you all, I learned from you that it's possible to have a callable module. Although `sys.modules[__name__] = wat` looks like a black magic and I'm a bit afraid of locking out to other importable things in this package, I think I'll go for it.
This looks super useful, but I was wondering if I'm the only one bothered by this recent trend of overloading completely unrelated operators (here the `/` operator) in the name of legibility.
Wow, this is the kind of tool that would have been a game changer when I was learning python; being able to see what is happening under the covers is part of the critical path for me to learn a language, and native python debugging is underwhelming, at best.
Instead I just installed pry and became a rabid ruby fanboy, but this might get me to give it another go.
There were various internal Quality of Life modules during my time at amzn, but nothing public. I suspect that given enough time working with Lua, each team will create an ad hoc, informally-specified, bug-ridden, slow implementation of half of python.
Python's biggest weakness is the lack of a function that shows you how exactly a variable looks like. the var_dump (php, ha!) works like wonders. The best library ever created for Python ;). and now wat is also looking to be a close contender for best helping hand.
Please don't do this. Exec'ing arbitrary, obfuscated code you just copied from the internet directly in a running environment is such a bad idea it shouldn't even be suggested.
At the very lease please put huge warnings around this section to let people know that it is a horrendously bad idea and they do it at their own peril.
How is it any different than installing the package via pip ? Not only most people won't check the source before running the code, but there is also no way to be sure that the code shipped by pip is the one you read on GitHub...
Yes, the big warning disclaimer in that part of the docs is definitely required here.
That idea of exec'ing arbitrary obfuscated (compressed) shell code that's easy to copy-paste into a python shell is very helpful, mind.
I've had to debug issues in production with silently hanging I/O threads and my only access is via `kubectl exec`.
This wat tool and that "insta-load" idea pairs nicely with pyrasite for that very useful !!!DO NOT DO THIS, YOUR CONTAINERS SHOULD BE LOCKED DOWN SO YOU CANNOT DO THIS!!! step of copying gdb and the pyrasite library into a running container when all you have is a python shell to play with.
(This almost feels like an exploit, running shellcode after getting RCE :))
Convenient way to quickly add extra debugging capability without rerunning. It isn't much different from the many `curl example.com/install.sh |bash` you see around. It's up to the user to check things out before running.
Yeah, the piping to bash is a tried and true method for various installers. People make a fuss about it, but we don’t see people getting owned that way often. I think with bash installers though it’s pretty trivial to just visit the link and read through the 100 lines of bash. So anything installed this way should be as simple as visiting the link and reading a short amount of code imo
Sure, but it's a few extra steps. It's been proven(tm) that many people prefer a single, simple thing they can just copy, paste and run, so they can get back to their main concern.
And some may not want it actually installed for whatever reason. Such as when there's no proper separation between dev and prod deps. (I'm mostly just guessing at this point though...)
Not everywhere that python is run has access to pip. Sshing into some locked down remote machine and needing to debug some script is a use case that comes to mind.
Unfortunately the Python part of that is not open source but it was only a few lines of code - should be easy to recreate. That repo does contain a C library that does a similar thing.
You might just say "why not just run your program directly with the debugger?" and yeah that is better when you can do it, but I'm working with complicated silicon verification flows that are often several layers of Python and Make, followed by a simulator (e.g. Questa or VCS) that itself loads the library you want to debug. Very difficult to make debugging work through all those layers, but it's quite easy to write some code at the bottom of the stack that says to a debugger "I'm here, debug me!".
It important to remind everyone that you can already do something similar via the build-in function help(). For example, if I run help({1}) I get the documentation for a set, running help(set.add) gives the documentation for the add() method, etc. You can even preview objects that you haven't imported by using strings(ex. running help('numpy.ndarray') will correctly give you the documentation, provided numpy is installed in your current python environment). It's pretty helpful and doesn't require installing anything extra
You're completely right. `dir()` does the same in terms of functionality. In fact, my tool makes use of `dir` under the hood. I just wanted to make it more readable, and to combine `dir`, `type`, `repr`, `len`, `inspect`, etc. into one easily accessible place.
Right, that would be a wrong pipe direction. Fun fact: It's already possible to do "wat | foo" or "wat << foo" (if you're more familiar with C++ iostream), it has the same effect as "wat / foo"
Looks like a great tool. Will start playing with it whenever I have to dive into an existing project again.
One thing I'd like to note, though, is that most engineers (at least around and including myself) would be triggered by the "Insta-load" example of executing base64 encoded (and thus obfuscated) code.
The insta-load is fine. You're supposed to save the snippet, decode and uncompress it, look at it to see what it does and that it's completely self-contained uncontroversial Python code and copy-paste your version that you bookmarked. It's super short.
just so that it is clear for everyone, including newbies here, does the insta-load code snippet work like this:
the author has encoded the wat module's source code into base64. the snippet shown decodes that back into python code, and then executes it on the fly, thus having the same effect as importing the wat module in the normal python way?
Yes, that has the same effect as installing the package. So if you feel uncomfortable, you can either install the pip package (and of course review the installed code) or review the decoded string before executing it. It's not that obfuscated anyways, it's still quite readable.
This is interesting. I'm wondering what compelled the author to use the division magic method for arguments instead of the more intuitive and commonly used approach to passing parameters.
I think it makes a lot of sense in the context of a live interpreter. Wrapping and object in help() or dir() is an annoying set of movements vs. just pressing 5 keys with no modifiers (Home, w,a,t,/). It makes its feel much more like a magic command without actually needing it to be magic. The dot modifiers are also convenient compared to passing kwargs imo.
Interesting choice indeed. It looks like it supports the traditional approach as well
> wat object can quickly inspect things by using the division operator (to avoid typing parentheses). A short, no-parentheses syntax wat / object is equivalent to wat(object).
It's the parentheses that drove me crazy. As people already noted, it's for faster typing, at the cost of the division magic as you noted. If it's more familiar to you, it works with `wat(object)` syntax as well.
I'm also really appreciating the base64 exec hack. All my years in Python I never thought of or came across it until now. I'll totally be using it for some things :).
[0] https://github.com/gabrielcnr/python-ls [1] https://pypi.org/project/snoop/
Surprised there isn't more innovation and new tools like this around python's interactive shell given it's one of the real strong points the language has.
1. Write the program skeleton
2. Put an import pdb; pdb.set_trace() whereever I need to fill out next
3. Once I hit the breakpoint, type 'interact'
4. Start coding in the debugger.
4a. For small functions write in a scratch pad, test against another python console, then paste into the current debugger.
4b. For network calls, save the inputs and outputs. Toss a decorator on some functions to log the inputs and outputs to files, and cool we already have some proto test data.
4c. For documentation, call help() on the variable or method or module.
If you never heard about it, scroll to the bottom to https://github.com/gruns/icecream#icecream-in-other-language...
Twenty years ago I wrote an object introspector for Zope ;)
Nowadays, I'm using devtools daily, and icecream and q occasionally. I'll give wat a try.
Given the cool nature of this project, I'm surprised they don't offer simply "import wat" with identical usage syntax. Thus inviting curious users to wat/wat in order to discover the trick...
Edit: Oh, you don't mean in the importing module's globals, you mean `sys.modules`. Yeah that works!
why do you say it's wonky? it's just
According to the official python documentation [0] of sys.modules:> This is a dictionary that maps module names to modules which have already been loaded. This can be manipulated to force reloading of modules and other tricks.
Thus, the modules dictionary is being used as intended, and it has the desired effect. Nothing wonky about that.
Of course, a __call__ method would be better because you could still keep the other functions inside the module. But for a single-function import like "wat" it seems quite natural and sane.
[0] https://docs.python.org/fr/3/library/sys.html
[0] https://pypi.org/project/snoop/
But I eventually printed that output, and put it in a dir I point my PYTHONPATH to so that I will always have it available.
Let's see if it sticks.
Instead I just installed pry and became a rabid ruby fanboy, but this might get me to give it another go.
see inspection.py in the wat module.
it has this on line 2:
import inspect as std_inspect
> Load it on the fly by pasting this snippet to your Python interpreter
The idea of having a project's readme include a full copy of the project itself as base64'd compressed data is pretty ingenious!
especially for a project like this where you may not have had the foresight to preload it into the environment where you most need it
--
1: https://github.com/Textualize/rich?tab=readme-ov-file#rich-p...
Doesn't the repr() built-in function do what you want?
It can also be customized by defining the __repr__() method of a class, IIRC.
Commonly done.
Read about repr() here:
https://docs.python.org/3/library/functions.html
and dunder repr, i.e. __repr__() here:
https://docs.python.org/3/reference/datamodel.html#object.__...
For node you it’s possible to hook it up with the browser inspector as well
Hooking devtools with Node is possible but could be a lot nicer.
I mean the number of times I've bumped on json not being able to dump the content of an object is just infuriating given how flexible Python is.
I'm definitely giving this a whirl.
Somthing like that does exist, and has, from Python 2.x; the inspect module, apart from help() and dir():
It is somewhat lower-level than wat, which builds upon it. But it is powerful.
https://news.ycombinator.com/item?id=41072878
Or did you mean the inspect module itself?
Please don't do this. Exec'ing arbitrary, obfuscated code you just copied from the internet directly in a running environment is such a bad idea it shouldn't even be suggested.
At the very lease please put huge warnings around this section to let people know that it is a horrendously bad idea and they do it at their own peril.
That idea of exec'ing arbitrary obfuscated (compressed) shell code that's easy to copy-paste into a python shell is very helpful, mind.
I've had to debug issues in production with silently hanging I/O threads and my only access is via `kubectl exec`.
This wat tool and that "insta-load" idea pairs nicely with pyrasite for that very useful !!!DO NOT DO THIS, YOUR CONTAINERS SHOULD BE LOCKED DOWN SO YOU CANNOT DO THIS!!! step of copying gdb and the pyrasite library into a running container when all you have is a python shell to play with.
(This almost feels like an exploit, running shellcode after getting RCE :))
https://pypi.org/project/pyrasite/
And some may not want it actually installed for whatever reason. Such as when there's no proper separation between dev and prod deps. (I'm mostly just guessing at this point though...)
In a previous company I set things up so that when there's an uncaught exception it will automatically start a VSCode debug session and connect to it.
Here's the extension: https://github.com/Timmmm/autodebug/
Unfortunately the Python part of that is not open source but it was only a few lines of code - should be easy to recreate. That repo does contain a C library that does a similar thing.
You might just say "why not just run your program directly with the debugger?" and yeah that is better when you can do it, but I'm working with complicated silicon verification flows that are often several layers of Python and Make, followed by a simulator (e.g. Questa or VCS) that itself loads the library you want to debug. Very difficult to make debugging work through all those layers, but it's quite easy to write some code at the bottom of the stack that says to a debugger "I'm here, debug me!".
Same reason jupyter notebooks allow you to prepend '?' or '??' to inspect a variable (though not expressions, in that respect this syntax is better)
I believe it should be equally possible to overload the bitwise OR operator (|), which, for people used to piping in Unix, is probably more intuitive.
One thing I'd like to note, though, is that most engineers (at least around and including myself) would be triggered by the "Insta-load" example of executing base64 encoded (and thus obfuscated) code.
the author has encoded the wat module's source code into base64. the snippet shown decodes that back into python code, and then executes it on the fly, thus having the same effect as importing the wat module in the normal python way?
with open('wat_source.py', 'w') as f: f.write(zlib.decompress(base64.b64decode(code.encode())).decode())
> wat object can quickly inspect things by using the division operator (to avoid typing parentheses). A short, no-parentheses syntax wat / object is equivalent to wat(object).