Tyrel's Blog

NOTES

This post is ported over from my wiki, so the format isn't as storytelling as a blog post could be, but I wanted it here.

Bill of Materials

• Raspberry Pi Zero W H (WiFi + Headers)
• BlinkT LED Strip GPIO

Home Assistant Parts

[Unit]
Description=Meeting Indicator

[Service]
Type=simple
ExecStart=/usr/bin/python2 /home/pi/mqtt.py
WorkingDirectory=/home/pi/Pimoroni/blinkt/examples
Restart=always
RestartSec=2

[Install]
WantedBy=sysinit.target

rgb,0,255,0,0
rgb,1,255,0,0
rgb,2,255,0,0
rgb,3,255,0,0
rgb,4,255,0,0
rgb,5,255,0,0
rgb,6,255,0,0
rgb,7,255,0,0

Images

Django ORM - My History

I'm not the best SQL developer, I know it's one of my weak points. My history is I did php/mysql from the early 2000s until college. In college I didn't really focus on the Database courses, the class selection didn't have many database course. The one Data Warehousing course I had available, I missed out on because I was in England doing a study abroad program that semester. My first job out of college was a Python/Django company - and that directed my next eight years of work.

Django, if you are unaware, is a MVC framework that ships with a really great ORM. You can do about 95% of your database queries automatically by using the ORM.

entry, created = Entry.objects.get_or_create(headline="blah blah blah")

q = Entry.objects.filter(headline__startswith="What")
q = q.filter(pub_date__lte=datetime.date.today())
q = q.exclude(body_text__icontains="food")

Above are some samples from the DjangoDocs. But enough about Django.

My Requirements

Recently at my job I was given a little bit of leeway on a project. My team is sort of dissolving and merging in with another team who already does Go. My Go history is building a CLI tool for the two last years of my previous job. I had never directly interacted with a database from Go yet. I wanted to spin up a REST API (I chose Go+Gin for that based on forty five seconds of Googling) and talk to a database.

GORM

Being that I come from the Django (and a few years of ActiveRecord) land, I reached immediately for an ORM, I chose GORM. If you want to skip directly to the source, check out https://gitea.tyrel.dev/tyrel/go-webservice-gin. Full design disclosure: I followed a couple of blog posts in order to develop this, so it is in the form explictly decided upon by the logrocket blog post and may not be the most efficient way to organize the module.

In order to instantiate a model definition, it's pretty easy. What I did is make a new package called models and inside made a file for my Album.

type Album struct {
      ID     string  `json:"id" gorm:"primary_key"`
      Title  string  `json:"title"`
      Artist string  `json:"artist"`
      Price  float64 `json:"price"`
}

This tracks with how I would do the same for any other kind of struct in Go, so this wasn't too difficult to do. What was kind of annoying was that I had to also make some structs for Creating the album and Updating the Album, this felt like duplicated effort that might have been better served with some composition.

I would have structured the controllers differently, but that may be a Gin thing and how it takes points to functions, vs pointers to receivers on a struct. Not specific to GORM. Each of the controller functions were bound to a gin.Context pointer, rather than receivers on an AlbumController struct.

The FindAlbum controller was simple:

func FindAlbum(c *gin.Context) {
      var album models.Album
      if err := models.DB.Where("id = ?", c.Param("id")).First(&album).Error; err != nil {
              c.JSON(http.StatusBadRequest, gin.H{"error": "Record not found!"})
      }
      c.JSON(http.StatusOK, gin.H{"data": album})
}

Which will take in a /:id path parameter, and the GORM part of this is the third line there.

models.DB.Where("id = ?", c.Param("id")).First(&album).Error

To run a select, you chain a Where on the DB (which is the connection here) and it will build up your query. If you want to do joins, this is where you would chain .Joins etc... You then pass in your album variable to bind the result to the struct, and if there's no errors, you continue on with the bound variable. Error handling is standard Go logic, if err != nil etc and then pass that into your API of choice (Gin here) error handler.

This was really easy to set up, and if you want to get a slice back you just use DB.Find instead, and bind to a slice of those structs.

var albums []models.Album
models.DB.Find(&albums)

SQLX

SQLX is a bit different, as it's not an ORM, it's extensions in Go to query with SQL, but still a good pattern for abstracting away your SQL to some dark corner of the app and not inline everywhere. For this I didn't follow someone's blog post — I had a grasp on how to use Gin pretty okay by now and essentially copied someone elses repo with my existing model. gin-sqlx-crud.

This one set up a bit wider of a structure, with deeper nested packages. Inside my internal folder there's controllers, forms, models/sql, and server. I'll only bother describing the models package here, as thats the SQLX part of it.

In the models/album.go file, there's your standard struct here, but this time its bound to db not json, I didn't look too deep yet but I presume that also forces the columns to set the json name.

type Album struct {
  ID     int64   `db:"id"`
  Title  string  `db:"title"`
  Artist string  `db:"artist"`
  Price  float64 `db:"price"`
}

An interface to make a service, and a receiver are made for applying the CreateAlbum form (in another package) which sets the form name and json name in it.

func (a *Album) ApplyForm(form *forms.CreateAlbum) {
  a.ID = *form.ID
  a.Title = *form.Title
  a.Artist = *form.Artist
  a.Price = *form.Price
}

So there's the receiver action I wanted at least!

Nested inside the models/sql/album.go file and package, is all of the Receiver code for the service. I'll just comment the smallest one, as that gets my point across. Here is where the main part of GORM/SQLX differ - raw SQL shows up.

func (s *AlbumService) GetAll() (*[]models2.Album, error) {
      q := `SELECT * FROM albums;`

      var output []models2.Album
      err := s.conn.Select(&output, q)
      // Replace the SQL error with our own error type.
      if err == sql.ErrNoRows {
              return nil, models2.ErrNotFound
      } else if err != nil {
              return nil, err
      } else {
              return &output, nil
      }
}

This will return a slice of Albums - but if you notice on the second line, you have to write your own queries. A little bit more in control of how things happen, with a SELECT * ... vs the gorm DB.Find style.

To me this feels more like using pymysql, in fact its a very similar process. (SEE NOTE BELOW) You use the service.connection.Get and pass in what you want the output bound to, the string query, and any parameters. This feels kind of backwards to me - I'd much rather have the order be: query, bound, parameters, but thats what they decided for their order.

Conclusion

Overall, both were pretty easy to set up for one model. Given the choice I would look at who the source code is written for. If you're someone who knows a lot of SQL, then SQLX is fine. If you like abstractions, and more of a "Code as Query" style, then GORM is probably the best of these two options.

I will point out that GORM does more than just "query and insert" there is migration, logging, locking, dry run mode, and more. If you want to have a full fledged system, that might be a little heavy, then GORM is the right choice.

SQLX is great if what you care about is marshalling, and a very quick integration into any existing codebase.

Repositories

• Go, Gin, Gorm
• Go, Gin, sqlx

Notes

I sent this blog post to my friend Andrey and he mentioned that I was incorrect with my comparision of sqlx to pymysql. To put it in a python metaphor, "sqlx is like using urllib3, gorm is like using something that generates a bunch of requests code for you. Using pymysql is like using tcp to do a REST request." Sqlx is more akin to SqlAlchemy core vs using SqlAlchemy orm. Sqlx is just some slight extensions over database/sql. As the sort of equivalent to pymysql in Go is database/sql/driver from the stdlib.