Tyrel's Blog

How to not trigger a post_save in Django, but still modify data.

Recently I have been diving into using signals with Django, which of course are pretty neat.

I am working on a website for work which in the most basicexplanation, is a task management site. Recently I have added in the ability to subscribe to tasks and get emails, I did this by connecting to the post_save signal. I only email out when a task is changed, not created (of course, no one would be subscribed to it). This worked flawlessly and "emails" out to anyone who is subscribed. I say that in quotes, because I haven't actually hooked it up to a real SMTP server, and only use

python -m smtpd -n -c DebuggingServer localhost:1025

which will output any emails to stdout. But I digress… A problem arose when I was working on ordering tasks.

I store an integer in the "ordering" column, which any authenticated user can drag the row to a new location and that will reorder the task. I did this after I setup the emailing signal, so I didn't think about an email being sent out for EVERY task being changed.

I tried a lot of different things, and was debating some that would be a bit messy. Among those ideas were trying to store the past values in another table, but that would get expensive fast. The reason I tried this was because I wanted to see if the ordering was the only thing that changed, and if that was the case, not send an email. I eventually found a thread on StackOverflow that says to use update on the queryset to not trigger the signal.

You can do this by doing something like this:

from app.models import ModelName

def reorder(request):
    new_order = request.POST.get('new_order', None)
    pk = request.POST.get('modelname_pk', None)
    if new_order:
       ModelName.objects.filter(pk=pk).update(ordering=new_order)

I am not sure if this is the proper way save changes and not trigger a post_save signal, but this is the way that worked for me so I figured I would document this.

Help, I have too many Django ManyToMany Queries [FIXED]

My boss tasked me with getting the load time of 90 seconds(HOLY CARP!) on one page down. First thing I did was install the Django Debug Toolbar to see what was really happening.

There are currently 2,000 users in the database, the way our model is setup is that a UserProfile can have other UserProfiles attached to it in one of three M2M relations, which in the Django Admin would cause 2,000 queries PER M2M field. This is very expensive as obviously you don't want 10,000 queries that each take 0.3ms to take place.

The solution, after a day and a half of research is to override the formfield_for_manytomany method in the Admin class for our UserProfile object.

Our solution is to prefetch for any M2M that are related to the current Model.

def formfield_for_manytomany(self, db_field, request, **kwargs):
    if db_field.__class__.__name__ == "ManyToManyField" and \
        db_field.rel.to.__name__ == self.model.__name__:
        kwargs['queryset'] = db_field.rel.to.objects.prefetch_related("user")
    return super(UserProfileInline, self).formfield_for_manytomany(
        db_field, request, **kwargs)

This goes inside our admin class UserProfileInline(admin.StackedInline). Simple clean and easy to drop into another ModelAdmin with minimal changes.

Other things I pondered was to set all our M2M's as raw_id_fields, then using Select2 or Chosen, query our UserProfiles when the related users were being selected. This would take a lot of load off the initial page load, but is more of a bandaid rather than a real fix.

I tried to override the Admin class's def queryset(self, request): but this was not affecting anything.

Getting started in Python Part 1

I have a friend who is interested in becoming a Python developer. He has some Python experience with CodeAcademy, but he of course wants to take this a step further and develop on his own computer. I figure I'd give him a few pointers, and I know this has been rehashed a million times, but what the hell, why not blog on it again. There are a few important things to learn besides the actual language itself. The first I am going to discuss is has to deal with installing packages, then followed up closely with Python's path trickery. Finally I'm going to wrap up by discussing some software related to development, that could be used for any language, but I use daily in my work as a Python Software Engineer. Let's get started.

$ sudo pip install virtualenv # Only time you might need sudo, try without first.
$ virtualenv myenv # Create the virtual environment
$ source myenv/bin/activate # Activate the virtual environment
(myenv)$ python -c "import MYPACKAGE; print MYPACKAGE"

#!/bin/bash
# This hook is run after every virtualenv is activated.
# if its a work or a personal project (Example)
proj_name=$(echo $VIRTUAL_ENV|awk -F'/' '{print $NF}')
if [[ -e "/Users/tsouza/PersonalProjects/$proj_name" ]]
then
    cd ~/PersonalProjects/$proj_name
else
    cd ~/WorkProjects/$proj_name
fi

Custom Django URLField

For work I had to write a custom url model field. This model field when setting up accepts a default protocol, and a list of other protocols.

When checking the protocol, the url is split by "://". If the split has one or two parts, then the url is validly formed.

In the event of a single element split, there is no protocol specified. When there is no protocol, the url is prepended with the default protocol specified. If there is a protocol, it is checked to make sure it exists in a union of the default protocol and other protocols. If it is not, a ValidationError is raised letting the user know that the protocol is not accepted.

This can all be found at On my github [deadlink].

I have a couple ways I could have done this better and probably will. Improvements would be just one parameter called parameters in which it is checked if there is at least one element. Passing this, when there is no protocol specified, the first element is the default one.

This would be a little cleaner.

this example would allow for http, https, ssh, spdy and mailto, anything else would error out.

facebook_page = URLField(default_protocol="http", protocols=["https","ssh","spdy","mailto"])

The way I could improve this would be

facebook_page = URLField(protocols=["https","https","ssh","spdy","mailto"])

Python Progress Bar

I was looking for a nice progress bar today at work to show progress rather than just printing "Waiting 30 seconds…" and having the script do nothing, I wanted to have a progress bar show.

I found a progress bar from Corey Goldberg

I did make a couple changes, and have uploaded my changes to my GitHub account.

newPythonProgressBar [deadlink]

To use this progressbar, it is very easy.

# To Setup
from progress_bar import ProgressBar
import sys
import time
def updateBar(step):
    p.update_time(step)
    sys.stdout.write("%s\r" % p)
    sys.stdout.flush()
#
# to call
#
wait_time = 100 # seconds
p = ProgressBar(wait_time)
p.empty_char = "."
p.unit = "^"
for step in range(wait_time+1):
    updateBar(step)
    time.sleep(1)

It will look like this when you use it

[###...............7%..................] 7^/100^