I never intended this to be a full fleshed CLI tool comparable to the likes of the real GitHub CLI. This was simply a way to refresh myself and have fun. I have accomplished this, and am now calling this "Feature Complete". You can play around with it yourself from the repository on gitlab.
HTTPX.LINKS
Today I learned some fun things with httpx mainly. The main thing I focused on today was figuring out pagination. The GitHub API uses the link header which I had never seen before.
The format of the header is a url, and then a relationship of that url. Lets take my friend Andrey's repos for example:
sample header value from Getting Andrey's repositories list and checking pagination
The link header there has two items split by a comma, each with two fields split by a semicolon, the first of which is a URL inside angle brackets... and AGHH this is going to be annoying to parse! Luckily httpx responses have this handled and client.get(...).links returns a lovely dictionary of the proper data.
With a response.links.get('next') check, you can get the url from response.links['next']['url']. So much nicer than writing some regular expressions.
TESTING
With that accomplished, I then added pytest-cov to my requirements.in and was able to leverage some coverage checks. I was about 30% with the latest changes (much higher than anticipated!) so I knew what I wanted to focus on next. The API seemed the easiest to test first again, so I changed around how I loaded my fixtures and made it pass in a name and open that file instead. In real code I would not have the function in both my test files, I would refactor it, but again, this is just a refresher, I'm lazy.
I decided earlier that I also wanted to catch HTTP 403 errors as I ran into a rate limit issue. Which, I assure you dear reader, was a thousand percent intentional so I would know what happens. Yeah, we'll go with that.
Py.Test has a context manager called pytest.raises and I was able to just with pytest.raises(httpx.HttpStatusError) and check that raise really easily.
The next bits of testing for the API were around the pagination, I faked two responses and needed to update my link header, checking the cases where there was NO link, was multiple pages, and with my shortcut return - in case the response was an object not a list. Pretty straight forward.
The GHub file tests were kind of annoying, I'm leveraging rich.table.Table so I haven't been able to find a nice "this will make a string for you" without just using rich's print function. I decided the easiest check was to see if the Table.Columns.Cells matched what I wanted, which felt a little off but it's fine.
The way I generated the table is by making a generator in a pretty ugly way and having a bunch of repo['column'],repo['column'] responses, rather than doing a dict comprehension and narrowing the keys down. If I ever come back to this, I MIGHT reassess that with a {k:v for k,v in repos if k in SELECTED_KEYS} and then yield a dictionary, but it's not worth the effort.
Overall I'd say this project was fun. It gave me a glimpse back into the Python world, and an excuse to write a couple blog posts. My next project is to get a Django site up and running again, so I can figure out how to debug my django-dbfilestorage.
Closing Thoughts
If I had to do this again, I would probably have tried some test driven development. I've tried in the past, but I don't work on a lot of greenfield projects. I tend to be the kind of engineer who jumps HEAD FIRST into code and then tests are an after thought.
I also kind of want to rewrite this in Go and Rust, two other languages I've been fond of lately, just to see how they'd compare in fun. I haven't done any API calls with Rust yet, only made a little Roguelike by following Herbert Wolverson's Hands-On-Rust book. The Tidelift CLI is all Go and a bazillion API calls (okay like ten) so that wouldn't be too hard to use like SPF13's Cobra CLI library and make a quick tool that way.
One fun thing I learned while moving things over to GitLab is that my user Tyrel is a super early adopter. I was in the first 36,000 people! I showed a screenshot of my user ID to my friend Sunanda at GitLab and we had fun finding that out.
It's no lie that I love terminals. I wish I could live on a terminal and never really need to see a GUI application again.
Last night I migrated a lot of my old code from one GitLab account to another (tyrelsouza to tyrel) in an effort to clean up some of my usernames spread across the world. While doing that I noticed my django-dbfilestorage Python module that has been sitting and rotting for three years. I played around a little bit in order to port it to Python 3.9, but I ran into some base64 errors. I tried a little bit but it was late and I couldn't figure it out. My resolve is that I have been away from Python for too long so the little things - that I knew and love - had fallen away. I mentioned this to my friend Alex and he said "make a barebones github cli (readonly?) with issue viewer, and stats display". I've embarked on a journey to refresh my Python well enough to repair DBFS.
I knew I wanted to use httpx as my network client library, it's new, fast, and I have a couple friends who work on it. I started with a barebones requirements.in file, tossed in invoke, pytest, and black. From there I used pip-compile to generate my requirements.txt - (a tip I picked up recently while adding Pip-Compile support to the Tidelift CLI) and I was good to go.
The docs for the GitHub API are pretty easy to read, so I knew all I really needed to do was set my Accept header to be Version3 and I could view the schema. With the schema saved to a .json file I then wrote a GHub class to pull this data down using httpx.client.Client.get, super simple! The only two endpoints I care about right now are the user and repos endpoints, so I made two get_ functions for each. After a little bit of work - which I won't bore you with the super intricate details - I have a functional cli.py file. For now, the only interaction is a propmt from rich for a username, and then you get a fancy table (also from rich) of the first page of results of repos, stargazer/watchers/forks counts, and a description.
Prompting for the username and showing my table of repositories.
It was a fun evening of learning what's changed in Python3 since I last touched it, especially as I've spent the majority of my career in Python2.7. Type annotations are super awesome. I'll probably pick it up again once I get some more free time later in the week. It's also nice blog fodder! I already have a million things I want to do next - pagination, caching, some more interaction.
Showing py.test running
I know the tool I'm writing is nothing special, especially with their own cli now, but I'm not looking at reinventing the wheel!
Check out the code so far on my GitLab (heh, ironic it's there).
A lot of times when I stop at someone's computer and help them in the terminal, I use a Readline command and people say "How the heck did you do that?"
Let me first backup and explain what Readline is. From the GNU Readline Documentation - "The GNU Readline library provides a set of functions for use by applications that allow users to edit command lines as they are typed in." By default, Readline is set up in Emacs mode, no you don't have to have an extra four fingers to use Readline, most of the commands are simple.
Here are a couple of the commands I use daily:
Movement
To move to the beginning of a line, you press C-a
To move to the end of a line you press C-e
Killing and Yanking
To cut the rest of the line from where your cursor is, to the end, you press C-k
To delete one word you press C-w
To paste either of the two previous back you can press C-y
Miscellaneous
To clear the screen and get to a fresh start, you can press C-l
To end your session you can send a C-d (This will send an end of file character)
To search for a command you typed recently, press C-r and start typing, it will search backwards. C-r again will search for an earlier match.
The inverse of C-r is C-s, they function the same.
To open your $EDITOR to edit the current shell command you wish to write, press C-xC-e
Finally, don't forget about C-c. While not specifically Readline, it's very useful because it sends the SIGINT signal to the program, which if just on the command line, will not execute the line you have type, and give you a new line with nothing on it. A nice clean start.
To find out a lot more, read the documentation at the Readline Commands Docs I even learned some things while writing this up, apparently pressing C-x $ will list off all the possible usernames. Good to know, and good to always keep learning.
