This is a quick example of SMS PIN verification using Twilio cloud services and Django which I did for a small site recently.
The page template contains the following HTML snippet
<input type="text" id="mobile_number" name="mobile_number" placeholder="111-111-1111" required>
<button class="btn" type="button" onClick="send_pin()"><i class="icon-share"></i> Get PIN</button>
and a JavaScript function utilizing jQuery:
function send_pin() {
$.ajax({
url: "{% url 'ajax_send_pin' %}",
type: "POST",
data: { mobile_number: $("#mobile_number").val() },
})
.done(function(data) {
alert("PIN sent via SMS!");
})
.fail(function(jqXHR, textStatus, errorThrown) {
alert(errorThrown + ' : ' + jqXHR.responseText);
});
}
Then create the following views in Django:
def _get_pin(length=5):
""" Return a numeric PIN with length digits """
return random.sample(range(10**(length-1), 10**length), 1)[0]
def _verify_pin(mobile_number, pin):
""" Verify a PIN is correct """
return pin == cache.get(mobile_number)
def ajax_send_pin(request):
""" Sends SMS PIN to the specified number """
mobile_number = request.POST.get('mobile_number', "")
if not mobile_number:
return HttpResponse("No mobile number", mimetype='text/plain', status=403)
pin = _get_pin()
# store the PIN in the cache for later verification.
cache.set(mobile_number, pin, 24*3600) # valid for 24 hrs
client = TwilioRestClient(settings.TWILIO_ACCOUNT_SID, settings.TWILIO_AUTH_TOKEN)
message = client.messages.create(
body="%s" % pin,
to=mobile_number,
from_=settings.TWILIO_FROM_NUMBER,
)
return HttpResponse("Message %s sent" % message.sid, mimetype='text/plain', status=200)
def process_order(request):
""" Process orders made via web form and verified by SMS PIN. """
form = OrderForm(request.POST or None)
if form.is_valid():
pin = int(request.POST.get("pin", "0"))
mobile_number = request.POST.get("mobile_number", "")
if _verify_pin(mobile_number, pin):
form.save()
return redirect('transaction_complete')
else:
messages.error(request, "Invalid PIN!")
else:
return render(
request,
'order.html',
{
'form': form
}
)
PINs are only numeric and are stored in the CACHE instead of the database which I think is simpler and less expensive in terms of I/O operations.
There are comments.
Difio is a Django based service which uses a profile
model to provide site-specific, per-user information.
In the process of open sourcing Difio its core
functionality becomes available as a Django app. The trouble is that the
UserProfile
model contains site-specific and proprietary data which doesn't
make sense to the public nor I want to release it.
The solution is to have a MockProfile
model and work with
that by default while www.dif.io and other implementations
override it as needed.
How do you do that without creating useless table and records in the database
but still have the profiles created automatically for every user?
It turns out the solution is quite simple. See my comments inside the code below.
class AbstractMockProfile(models.Model):
"""
Any AUTH_PROFILE_MODULE model should inherit this
and override the default methods.
This model provides the FK to User!
"""
user = models.ForeignKey(User, unique=True)
def is_subscribed(self):
""" Is this user subscribed for our newsletter? """
return True
class Meta:
# no DB table created b/c model is abstract
abstract = True
class MockProfileManager(models.Manager):
"""
This manager creates MockProfile's on the fly without
touching the database. It is needed by User.get_profile()
b/c we can't have an abstract base class as AUTH_PROFILE_MODULE.
"""
def using(self, *args, **kwargs):
""" It doesn't matter which database we use! """
return self
def get(self, *args, **kwargs):
"""
User.get_profile() calls .using(...).get(user_id__exact=X)
so we instrument it here to return a MockProfile() with
user_id=X parameter. Anything else may break!!!
"""
params = {}
for p in kwargs.keys():
params[p.split("__")[0]] = kwargs[p]
# this creates an object in memory. To save it to DB
# call obj.save() which we DON'T do anyway!
return MockProfile(params)
class MockProfile(AbstractMockProfile):
"""
In-memory (fake) profile class used by default for
the AUTH_PROFILE_MODULE setting.
"""
objects = MockProfileManager()
class Meta:
# DB table is NOT created automatically
# when managed = False
managed = False
In Difio core the user profile is always used like this
profile = request.user.get_profile()
if profile.is_subscribed():
pass
and by default
AUTH_PROFILE_MODULE = "difio.MockProfile"
Voila!
There are comments.
I wasn't able to find detailed information on how to skip rendering or only render specific blocks from Jinja2 templates so here's my solution. Hopefully you find it useful too.
With below template I want to be able to render only kernel_options block as a single line and then render the rest of the template excluding kernel_options.
{% block kernel_options %}
console=tty0
{% block debug %}
debug=1
{% endblock %}
{% endblock kernel_options %}
{% if OS_MAJOR == 5 %}
key --skip
{% endif %}
%packages
@base
{% if OS_MAJOR > 5 %}
%end
{% endif %}
To render a particular block you have to use the low level Jinja API template.blocks. This will return a dict of block rendering functions which need a Context to work with.
The second part is trickier. To remove a block we have to create an extension which will filter it out. The provided SkipBlockExtension class does exactly this.
Last but not least - if you'd like to use both together you have to disable caching in the Environment (so you get a fresh template every time), render your blocks first, configure env.skip_blocks and render the entire template without the specified blocks.
#!/usr/bin/env python
import os
import sys
from jinja2.ext import Extension
from jinja2 import Environment, FileSystemLoader
class SkipBlockExtension(Extension):
def __init__(self, environment):
super(SkipBlockExtension, self).__init__(environment)
environment.extend(skip_blocks=[])
def filter_stream(self, stream):
block_level = 0
skip_level = 0
in_endblock = False
for token in stream:
if (token.type == 'block_begin'):
if (stream.current.value == 'block'):
block_level += 1
if (stream.look().value in self.environment.skip_blocks):
skip_level = block_level
if (token.value == 'endblock' ):
in_endblock = True
if skip_level == 0:
yield token
if (token.type == 'block_end'):
if in_endblock:
in_endblock = False
block_level -= 1
if skip_level == block_level+1:
skip_level = 0
if __name__ == "__main__":
context = {'OS_MAJOR' : 5, 'ARCH' : 'x86_64'}
abs_path = os.path.abspath(sys.argv[1])
dir_name = os.path.dirname(abs_path)
base_name = os.path.basename(abs_path)
env = Environment(
loader = FileSystemLoader(dir_name),
extensions = [SkipBlockExtension],
cache_size = 0, # disable cache b/c we do 2 get_template()
)
# first render only the block we want
template = env.get_template(base_name)
lines = []
for line in template.blocks['kernel_options'](template.new_context(context)):
lines.append(line.strip())
print "Boot Args:", " ".join(lines)
print "---------------------------"
# now instruct SkipBlockExtension which blocks we don't want
# and get a new instance of the template with these blocks removed
env.skip_blocks.append('kernel_options')
template = env.get_template(base_name)
print template.render(context)
print "---------------------------"
The above code results in the following output:
$ ./jinja2-render ./base.j2
Boot Args: console=tty0 debug=1
---------------------------
key --skip
%packages
@base
---------------------------
Teaser: this is part of my effort to replace SNAKE with a client side kickstart template engine for Beaker so stay tuned!
There are comments.
While working on open-sourcing Difio I needed to remove
all hard-coded URL references from the templates. My solution was to essentially
inherit from the standard {% url %}
template tag. Here is how to do it.
Difio is not hosted on a single server. Parts of the website are static HTML, hosted on Amazon S3. Other parts are dynamic - hosted on OpenShift. It's also possible but not required at the moment to host at various PaaS providers for redundancy and simple load balancing.
As an easy fix I had hard-coded some URLs to link to the static S3 pages and others go link to my PaaS provider.
I needed a simple solution which can be extended to allow for multiple domain hosting.
The solution I came up with is to override the standard {% url %}
tag and use it everywhere in my templates. The new tag will produce absolute URLs containing
the specified protocol plus domain name and view path. For this to work you have to
inherit the standard URLNode
class and override the render()
method to include the new
values.
You also need to register a tag method to utilize the new class. My approach was to use
the existing url()
method to do all background processing and simply casting the result
object to the new class.
All code is available at https://djangosnippets.org/snippets/3013/.
To use in your templates simply add
{% load fqdn_url from fqdn_url %}
<a href="{% fqdn_url 'dashboard' %}">Dashboard</a>
There are comments.
Recently I wrote about my problem with duplicate Amazon SQS messages causing multiple emails for Difio. After considering several options and feedback from @Answers4AWS I wrote a small decorator to fix this.
It uses the cache backend to prevent the task from executing twice during the specified time frame. The code is available at https://djangosnippets.org/snippets/3010/.
As stated on Twitter you should use Memcache (or ElastiCache) for this.
If using Amazon S3 with my
django-s3-cache don't use the
us-east-1
region because it is eventually consistent.
The solution is fast and simple on the development side and uses my existing cache infrastructure so it doesn't cost anything more!
There is still a race condition between marking the message as processed and the second check but nevertheless this should minimize the possibility of receiving duplicate emails to an accepted level. Only time will tell though!
There are comments.
How do you keep state when working with a stateless protocol like HTTP? One possible answer is to use a cache back-end.
I'm working on an IVR application demo with Django and Twilio. The caller will make multiple choices using the phone keyboard. All of this needs to be put together and sent back to another server for processing. In my views I've added a simple cache get/set lines to preserve the selection.
Here's how it looks with actual application logic omitted
def incoming_call(request):
call_sid = request.GET.get('CallSid', '')
caller_id = request.GET.get('From', '')
state = {'from' : caller_id}
cache.set(call_sid, state)
return render(request, 'step2.xml')
def step2(request):
call_sid = request.GET.get('CallSid', '')
selection = int(request.GET.get('Digits', 0))
state = cache.get(call_sid, {})
state['step2_selection'] = selection
cache.set(call_sid, state)
return render(request, 'final_step.xml')
def final_step(request):
call_sid = request.GET.get('CallSid', '')
selection = int(request.GET.get('Digits', 1))
state = cache.get(call_sid, {})
state['final_step_selection'] = selection
Backend.process_user_selections(state)
return render(request, 'thank_you.xml')
At each step Django will update the current state associated with this call and return
a TwiML XML response. CallSid
is a handy unique
identifier provided by Twilio.
I am using the latest django-s3-cache version which properly works with directories. When going to production that will likely switch to Amazon ElastiCache.
There are comments.
In case you are wondering how to use Django's built-in template tags and filters in your source code, not inside a template here is how:
>>> from django.template.defaultfilters import *
>>> filesizeformat(1024)
u'1.0 KB'
>>> filesizeformat(1020)
u'1020 bytes'
>>> filesizeformat(102412354)
u'97.7 MB'
>>>
All built-ins live in pythonX.Y/site-packages/django/template/defaultfilters.py
.
There are comments.
Did you ever have to re-purpose a column in your database schema? Here's a quick and easy way to do this if you happen to be using Django.
I had an integer field in my model called lines
which counted the lines of
code in a particular tar.gz package. I figured the file size is a better indicator
so decided to start using it. I was not planning to use the old field anymore and
I didn't care about the data it was holding. So I decided to re-purpose it
as the size
field.
Looking around I figured several different ways to do this:
lines
field and keep referencing the old name in the code.
This is no-brainer but feels awkward and is a disaster waiting to happen;size
field and remove the old lines
field. This involves modification to
the DB schema and requires at least a backup with possible down time. Not something
I will jump at right away;size
property in the model class which will persist to self.lines
.
This is a quick way to go but I'm not sure if one can use the property with the
Django QuerySet API (objects.filter(), objects.update(), etc.) I suspect not.
If you don't filter by the property or use it in bulk operations this method is fine though;size
but continue to use the lines
DB column;
Mind my wording here :);I decided to go for option 4 above:
change the field name to size
but continue to use the lines
DB column.
diff --git a/models.py b/models.py
index e06d2b2..18cad6f 100644
--- a/models.py
+++ b/models.py
@@ -667,7 +667,7 @@ class Package(models.Model):
- lines = models.IntegerField(default=None, null=True, blank=True)
+ size = models.IntegerField(default=None, null=True, blank=True, db_column='lines')
lines
from the code except the model class. This served as clean up as well. size
but continued using the lines
DB column as shown above.
Django's db_column option makes this possible.size
to None (NULL) for all objects;size
field.The entire process happened for under 10 minutes. I will also opt for renaming the DB column at a later time. This is to sync the naming used in Python code and in MySQL in case I ever need to use raw SQL or anything but Django.
If you were me, how would you do this? Please share in the comments below.
There are comments.
How do you order Django QuerySet results so that first item is the
exact match if using contains
or icontains
? Both solutions were proposed on the
django-users
mailing list.
Solution by Tom Evans, example is mine:
>>> from django.db.models import Q
>>> Package.objects.filter(
Q(name=Django) | Q(name__icontains=Django)
).extra(
select={'match' : 'name = "Django"'}
).order_by('-match', 'name')
[<Package: Django>, <Package: appomatic_django_cms>, <Package: appomatic_django_filer>,
<Package: appomatic_django_vcs>, <Package: BabelDjango>, <Package: BDD4Django>,
<Package: blanc-django-admin-skin>, <Package: bootstrap-django-forms>,
<Package: capistrano-django>, <Package: ccnmtldjango>, <Package: collective.django>,
<Package: csdjango.contactform>, <Package: cykooz.djangopaste>,
<Package: cykooz.djangorecipe>, <Package: d51.django.virtualenv.test_runner>,
<Package: django-4store>, <Package: django-503>, <Package: django-absolute>,
<Package: django-abstract-templates>, <Package: django-account>,
'...(remaining elements truncated)...']
>>>
Another one:
I'm not sure this is the right way, but you could drop the Q objects, use only icontains and sort by the length of 'name'
Gabriel https://groups.google.com/d/topic/django-users/OCNmIXrRgag/discussion
>>> packages = [p.name for p in Package.objects.filter(name__icontains='Dancer')]
>>> sorted(packages, key=len)
[u'Dancer', u'Dancer2', u'breakdancer', u'Task::Dancer', u'App::Dancer2', u'Dancer::Routes',
u'DancerX::Routes', u'DancerX::Config', u'Task::DWIM::Dancer', u'Dancer::Plugin::CDN',
u'Dancer::Plugin::Feed', u'Dancer::Plugin::LDAP', u'Dancer::Plugin::Lucy',
'...(remaining elements truncated)...']
>>>
That's all folks. If you have other more interesting sorting needs please comment below. Thanks!
There are comments.
Every now and then users forget their passwords. This is why I prefer using OAuth and social network accounts like GitHub or Twitter. But what do you do when somebody forgets which OAuth provider they used to login to your site? Your website needs a reminder. This is how to implement one if using django-social-auth.
Create a similar view on your Django back-end
def ajax_social_auth_provider_reminder(request):
"""
Remind the user which social auth provider they used to login.
"""
if not request.POST:
return HttpResponse("Not a POST", mimetype='text/plain', status=403)
email = request.POST.get('email', "")
email = email.strip()
if not email or (email.find("@") == -1):
return HttpResponse("Invalid address!", mimetype='text/plain', status=400)
try:
user = User.objects.filter(email=email, is_active=True).only('pk')[0]
except:
return HttpResponse("No user with address '%s' found!" % email, mimetype='text/plain', status=400)
providers = []
for sa in UserSocialAuth.objects.filter(user=user.pk).only('provider'):
providers.append(sa.provider.title())
if len(providers) > 0:
send_templated_mail(
template_name='social_provider_reminder',
from_email='Difio <reminder@dif.io>',
recipient_list=[email],
context={'providers' : providers},
)
return HttpResponse("Reminder sent to '%s'" % email, mimetype='text/plain', status=200)
else:
return HttpResponse("User found but no social providers found!", mimetype='text/plain', status=400)
This example assumes it is called via POST request which contains the email address. All responses are handled at the front-end via JavaScript. If a user with the specified email address exists this address will receive a reminder listing all social auth providers associated with the user account.
On the browser side I like to use Dojo. Here is a simple script which connects to a form and POSTs the data back to the server.
require(["dojo"]);
require(["dijit"]);
function sendReminderForm(){
var form = dojo.byId("reminderForm");
dojo.connect(form, "onsubmit", function(event){
dojo.stopEvent(event);
dijit.byId("dlgForgot").hide();
var xhrArgs = {
form: form,
handleAs: "text",
load: function(data){alert(data);},
error: function(error, ioargs){alert(ioargs.xhr.responseText);}
};
var deferred = dojo.xhrPost(xhrArgs);
});
}
dojo.ready(sendReminderForm);
You can try this out at Difio and let me know how it works for you!
There are comments.
I have been experimenting with the twitter module for Python and decided to combine it with django-social-auth to welcome new users who join Difio. In this post I will show you how to tweet on behalf of the user when they join your site and send them a welcome email.
In django-social-auth the authentication workflow is handled by an operations pipeline where custom functions can be added or default items can be removed to provide custom behavior. This is how our pipeline looks:
SOCIAL_AUTH_PIPELINE = (
'social_auth.backends.pipeline.social.social_auth_user',
#'social_auth.backends.pipeline.associate.associate_by_email',
'social_auth.backends.pipeline.user.get_username',
'social_auth.backends.pipeline.user.create_user',
'social_auth.backends.pipeline.social.associate_user',
'social_auth.backends.pipeline.social.load_extra_data',
'social_auth.backends.pipeline.user.update_user_details',
'myproject.tasks.welcome_new_user'
)
This is the default plus an additional method at the end to welcome new users.
You also have to create and configure a Twitter application so that users can login with Twitter OAuth to your site. RTFM for more information on how to do this.
This is how the custom pipeline action should look:
from urlparse import parse_qs
def welcome_new_user(backend, user, social_user, is_new=False, new_association=False, *args, **kwargs):
"""
Part of SOCIAL_AUTH_PIPELINE. Works with django-social-auth==0.7.21 or newer
@backend - social_auth.backends.twitter.TwitterBackend (or other) object
@user - User (if is_new) or django.utils.functional.SimpleLazyObject (if new_association)
@social_user - UserSocialAuth object
"""
if is_new:
send_welcome_email.delay(user.email, user.first_name)
if backend.name == 'twitter':
if is_new or new_association:
access_token = social_user.extra_data['access_token']
parsed_tokens = parse_qs(access_token)
oauth_token = parsed_tokens['oauth_token'][0]
oauth_secret = parsed_tokens['oauth_token_secret'][0]
tweet_on_join.delay(oauth_token, oauth_secret)
return None
This code works with django-social-auth==0.7.21 or newer. In older versions the
new_association
parameter is missing as
I discovered.
If you use an older version you won't be able to distinguish between newly created
accounts and ones which have associated another OAuth backend. You are warned!
Sending the welcome email is out of the scope of this post. I am using django-templated-email to define how emails look and sending them via Amazon SES. See Email Logging for Django on RedHat OpenShift With Amazon SES for more information on how to configure emailing with SES.
Here is how the Twitter code looks:
import twitter
from celery.task import task
from settings import TWITTER_CONSUMER_KEY, TWITTER_CONSUMER_SECRET
@task
def tweet_on_join(oauth_token, oauth_secret):
"""
Tweet when the user is logged in for the first time or
when new Twitter account is associated.
@oauth_token - string
@oauth_secret - string
"""
t = twitter.Twitter(
auth=twitter.OAuth(
oauth_token, oauth_secret,
TWITTER_CONSUMER_KEY, TWITTER_CONSUMER_SECRET
)
)
t.statuses.update(status='Started following open source changes at http://www.dif.io!')
This will post a new tweet on behalf of the user, telling everyone they joined your website!
NOTE:
tweet_on_join
and send_welcome_email
are Celery tasks, not ordinary Python
functions. This has the advantage of being able to execute these actions async
and not slow down the user interface.
Are you doing something special when a user joins your website? Please share your comments below. Thanks!
There are comments.
Common functionality for websites is the 'DELETE ACCOUNT' or 'DISABLE ACCOUNT' button. This is how to implement it if using django-social-auth.
delete_objects_for_user(request.user.pk) # optional
UserSocialAuth.objects.filter(user=request.user).delete()
User.objects.filter(pk=request.user.pk).update(is_active=False, email=None)
return HttpResponseRedirect(reverse('django.contrib.auth.views.logout'))
This snippet does the following:
User
object. You could also delete it but mind foreign keys;User
object - if a new user is created after deletion
we don't want duplicated email addresses in the database;There are comments.
Sending email in the cloud can be tricky. IPs of cloud providers are blacklisted because of frequent abuse. For that reason I use Amazon SES as my email backend. Here is how to configure Django to send emails to site admins when something goes wrong.
# Valid addresses only.
ADMINS = (
('Alexander Todorov', 'atodorov@example.com'),
)
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'handlers': {
'mail_admins': {
'level': 'ERROR',
'class': 'django.utils.log.AdminEmailHandler'
}
},
'loggers': {
'django.request': {
'handlers': ['mail_admins'],
'level': 'ERROR',
'propagate': True,
},
}
}
# Used as the From: address when reporting errors to admins
# Needs to be verified in Amazon SES as a valid sender
SERVER_EMAIL = 'django@example.com'
# Amazon Simple Email Service settings
AWS_SES_ACCESS_KEY_ID = 'xxxxxxxxxxxx'
AWS_SES_SECRET_ACCESS_KEY = 'xxxxxxxx'
EMAIL_BACKEND = 'django_ses.SESBackend'
You also need the django-ses dependency.
See http://docs.djangoproject.com/en/dev/topics/logging for more details on how to customize your logging configuration.
I am using this configuration successfully at RedHat's OpenShift PaaS environment. Other users have reported it works for them too. Should work with any other PaaS provider.
There are comments.