Posted Sat Feb 04 @ 10:32:03 AM PDT 2012
By default, Django sets the domain of a cookie to be the domain in which it was created. For example, if a cookie is created on example.com, that cookie is only available to example.com. Seems pretty reasonable to me.
There is a subtle issue with this, however. If your site is available from example.com, and www.example.com, and the user switches between them (either on their own, or because of a link on the site), they will effectively lose the cookie. So if they are logged in, they have to log in again as soon as they switch domains.
To fix this, you can add this line to your settings.py file:
SESSION_COOKIE_DOMAIN = ".example.com"
where example.com is your domain name. This will allow the cookie to float freely between any subdomain on your domain.
But this actually doesn't solve the real problem. You shouldn't allow users to use the www and non-www version of your website. You should redirect one to the other. Maybe I will write about how to fix that next time...
Posted Sat Jan 28 @ 10:51:07 AM PDT 2012
I recently discovered that MySQL does not do any compile time checking on queries with derived tables.
I use derived tables a lot for reporting queries - queries that take a long time to run. It is annoying when there is a simple bug in the query (like a typo in a column name), and MySQL doesn't catch it until the query has almost finished executing. If you're not familiar with the concept of derived tables, here's an example (from MySQL's website).
Suppose that you want to know the average of a set of sums for a grouped table. This does not work:
SELECT AVG(SUM(column1)) FROM t1 GROUP BY column2;
But with a derived table, you can get it:
SELECT AVG(sum_column1)
FROM (SELECT SUM(column1) AS sum_column1
FROM t1 GROUP BY column2) AS t1;
In my case, for reporting queries, a derived table might take minutes to evaluate. When you have a query of the form:
SELECT column_that_does_not_exist FROM (Really Long Running Query) k
MySQL will execute the code for the derived table first, without determining if the columns in the outer query are even possible to get.
Posted Sat Jan 14 @ 12:18:15 PM PDT 2012
I have multiple shell windows open (using PuTTY of course) all the time. Typically, they are shells for my development box. But when I need to make some changes to the production server, I will have a shell open for that.
Unfortunately, it is really easy to start typing commands into the wrong shell window, without realizing it (especially since my development and productions boxes are almost identical in their configurations). On more than one occasion, I have done something to the production machine when I really meant to do it on my dev box.
To prevent that from happening again, I changed the background color of my shell on the production machine to this awful red color.
Now I don't have any trouble determining which machine I'm on.
Posted Wed Jan 04 @ 10:29:24 AM PDT 2012
I installed fail2ban on my Ubuntu server, and after testing it, I determined that it wasn't actually banning anyone who tried to brute force an SSH password.
The problem is that /var/log/auth.log compresses duplicate messages. Here is a sample from my auth.log
Jan 2 13:54:27 delta sshd[16882]: Failed password for root from 175.210.205.169 port 4303 ssh2
Jan 2 13:54:35 delta sshd[16882]: last message repeated 3 times
Notice the "repeated 3 times" part. Fail2ban isn't capable of parsing that (at least Fail2Ban v0.8.4).
To fix the problem, you need to edit /etc/rsyslog.conf and set "$RepeatedMsgReduction" to "off". That will make auth.log explicitly log the whole message again, so fail2ban can parse it.
You also need to restart the rsyslog process:
service rsyslog restart
for the change to take effect.
Posted Mon Jan 02 @ 01:09:17 PM PDT 2012
After launching Bingo Baker, I decided to consolidate my web applications onto a single server, well, a VPS from Linode. One of my goals was to figure out a way to do database backups without punishing the server. My old way of doing backups was pretty bad.
I would do a mysqldump with the --lock-tables option set to false, so inserts and updates could still happen on my MyISAM tables. The dump would also be saved to the same machine being backed up, which makes the disk do double duty (read from the database on disk, then write it back to the disk). After the SQL dump was generated, I would transfer it to another machine.
The main problems with that backup scheme are:
- You get an inconsistent backup because the tables can change while the backup is occurring
- Your CPU, disk and memory work hard to generate the backup, and your website visitors suffer from the slow down
- You use up a lot of bandwidth transferring a gzipped database dump to another machine
On my new machine, I did the smarter thing, and setup a MySQL master-slave pair (with the master being the new server, and the slave being an old computer I had laying around in my house). It was incredibly easy to setup (change a few settings in /etc/mysql/my.cnf and you're good to go). You can learn how to do replication on MySQL here.
The cool part about replication is that I can stop the slave, do a mysqldump, and start it back up, to get a consistent snapshot of the master. During the backup, the master can continue to do write queries. When the slave comes back online, it can catch up to the master. I also don't have to hammer the master with any heavy IO work.
Another benefit, is that I can execute long running read queries for reports on the slave, and I don't have to tie up the master.
I wish I had done this years ago.
Posted Wed Dec 28 @ 12:35:07 PM PDT 2011
On the bingo web application I'm building, one of the problems I had to solve was how to determine if a string of text would fit into a n by n pixel box. If the text didn't fit, then I needed to shrink its font size until it did. So after doing some research, I found a method that works in most browsers (IE 7 and up, Firefox, Chrome and Safari). It doesn't work so well in Opera.
The Setup
You need two "testing" elements on your page. One is used to determine if the text fits into the box vertically (the textarea), the other is used to determine if the longest word can fit horizontally (the span).
And of course, you need one element to hold the input (the div).
For the CSS, you define the 'cell' class with a width and height of the square element that you want the text to fit into, and the font type and size:
You also need some simple CSS for the testing elements. The padding acts as padding for the input cell (since it forces the font-size to shrink even more) and the overflow:hidden makes the vertical scrollbar on the textarea disappear so it doesn't interfere with the width calculations.
The JavaScript (requires jQuery)
The function setFontSize, takes a jQuery element and determines the font size that makes the text in that element fit into the box specified by the element's CSS height and width.
There are two helper functions, getLongestWord and hasScrollbar. getLongestWord() takes as an argument the cell element and returns the longest whitespace delimited word in the cell.
hasScrollerbar() takes a regular DOM element, and determines if it has a scrollbar. In this case, we want to know if the textarea has a scrollbar.
To execute this code do something like this:
You can see a demo here.
Posted Wed Dec 21 @ 04:36:14 PM PDT 2011
The little project I started two weeks ago has now grown into a full blown application.
Instead of relying on all those pipes and little programs, I wrote a server in Python to listen for incoming log files and parse out the IP and hostname. I also added a little code to parse "money" messages, so I can detect when any of my sites make money (and I display that on the map).
I used the Google Maps API color schemer to make a cool black and gray map.
I put the code for Gstatus up on Github, so you can check it out. I tried to write a thorough README to get you bootstrapped, if you want to set it up for your websites.
Posted Sat Dec 10 @ 06:05:23 PM PDT 2011
Instead of studying for my finals, I played around with Google's Map API. I thought it would be cool to plot my website visitors on a map in near real time. I ended up using a combination of linux utilities, Perl, C, PHP, JavaScript and MySQL to achieve the desired effect.
Getting IP and Host from Apache
The first thing I needed to do was extract the visitor's IP and host (i.e. which website they were viewing on my server that hosts multiple websites). Apache's log file is the natural place to look for that information. Unfortunately, by default, Apache does not log the host. But it is easy to fix. All I needed to do was add %{Host}i to the LogFormat section:
If you follow the Apache log, you will see everything you need:
The reason I use --follow=name is because Apache swaps out the log file every once in a while, and I want to follow that filename, not the inode currently associated with the filename.
Sending Data Over Network
Here's the cool part. There is a linux command that makes it really easy to pipe stuff over a TCP connection (or UDP, if you like to live on the edge). It's called nc. Want to send that log file over the network? Piece of cake:
As long as you have nc listening on port 5000 on the other machine:
the log file will be piped out over the network onto the listening machine. Pretty cool.
Extracting IP and Host From Log (incoming over the network)
On the machine receiving the log, send the data to Perl to parse out the interesting parts:
That $|=1 code is used to disable buffering STDOUT (I don't want Perl to buffer the output, I want to see it right away).
Getting Latitude and Longitude from IP
The generous folks at MaxMind offer a free binary file (and C bindings) that map an IP address to a lat and long. It installs without a hitch on linux. I wrote a quick program, based on one of their examples, that would get the coordinates for the IP I parsed out in Perl:
Saving IP, Host, Lat and Long
I'll admit, this isn't the best way to store this streaming data, but it was pretty simple. I created a new database called IpQueue, and created a single table:
As my IP, host, lat and long tuples come out of my C program, I spit out an insert query (with an update):
Then I pipe that query to MySQL:
The entire pipe looks like this (it's a mess):
And we're still not done...
Outputting Points in HTTP
I made a simple page in PHP that outputs the most recent points stored in the database:
Google Maps
And finally we get to the Google map code. I simply query the PHP page roughly every second, and plot new points, and remove the old ones:
The end result is like this (I wish I could make this into an animation, but I don't know how):
Now I can study for my finals...
Posted Tue Dec 06 @ 08:37:38 PM PDT 2011
Nothing is more mystifying
than a proof of the obvious.
- Paul Lockhart in A Mathematician's Lament (PDF)
Posted Sat Dec 03 @ 02:06:21 PM PDT 2011
I recently upgraded the RAM on my computer from 8 gigs, to 16 gigs (which I'll admit, was frivolous, because I don't even come close to using 8 gigs). After I installed the memory, I noticed that nearly all the free space on my hard drive disappeared (I have a 120 gig SSD).
After snooping around my hard drive without finding any suspicious files, I changed my folder options to show hidden operating system files. I immediately noticed to huge files:
- pagefile.sys
- hiberfil.sys
both were approximately 16 gigabytes in size.
pagefile.sys
According to Microsoft, it is safe to reduce the size of your pagefile.sys if you have plenty of RAM (the article mentions 64 MB, but I think that is a bit dated):
if you have enough surplus RAM to support the operating system and application memory requirements, the need for a large paging file is lessened. If you have, for example, 64 MB of RAM, you may be able to safely reduce your paging file to a very small size.
You can shrink your pagefile by going to Control Panel > Search for "advanced" > "View advanced system settings" > Advanced > Settings (under the Performance group) > Advanced > Change...
I changed mine to have an initial size of 16, and a Maximum size of 1024. Make sure you click the "Set" button before hitting OK.
hiberfil.sys
This file keeps a copy of your RAM when the computer goes into hibernate. The only way to get rid of this file is to disable hibernation (which you shouldn't do on a laptop, since it saves power).
If you bring up cmd (make sure you run it as an administrator), simply type:
powercfg -h off
to turn off hibernation.
Those tricks reclaimed most of the hard drive space I lost when I upgraded my RAM.
Newer Posts
Older Posts