I just spent a while banging my head against activerecord. All my relations seemed right, but I kept getting the same error:


>> User.first.roles
ActiveRecord::HasManyThroughAssociationNotFoundError: Could not find the association "user_roles" in model User


Finally, I noticed that in my model, I had used a string to specify the :through value. Use a symbol!

Essentially, code that had to change was:

class User < ActiveRecord::Base
has_many :user_roles
has_many :roles, :through => "user_roles"

to:

class User < ActiveRecord::Base
has_many :user_roles
has_many :roles, :through => :user_roles

A frustrating error, but easily solved by using a symbol to do a symbol’s work.

If you want to see what your mysql server is working on – the current query, for example – there is an easy way to do it. Open your mysql prompt. This should be something like:

mysql -u root -p

then type this command:

SHOW PROCESSLIST;

You’ll then get a table listing what your server is working on at the moment. One of the entries will probably be processlist itself.

+-----+------+-----------------+------+---------+------+--------------+----------------------------+
| Id  | User | Host            | db   | Command | Time | State        | Info                       |
+-----+------+-----------------+------+---------+------+--------------+----------------------------+
| 497 | root | localhost:58057 | lms  | Sleep   |  111 |              | NULL                       |
| 508 | root | localhost       | NULL | Query   |    0 | NULL         | show processlist           |
| 509 | root | localhost       | qrs  | Query   |   23 | Sending data | select count(*) from users |
+-----+------+-----------------+------+---------+------+--------------+----------------------------+

You can kill a process that is taking too long or consuming too many resources. Just kill the number of the process (ID).

KILL 509

I am no longer out of a job. I am starting a new gig on Monday. After that I will be officially “in” a job. This is exciting!

I’m building a site for a record label I’m starting with some of the guys in my band. I imagine this will be an ongoing project for several months. I’m building it in Zend.

There are three basic types of users: consumers, bands, and admins (employees of the label). I decided the other day that a member of the admin team should be allowed to masquerade as any other user in the system without needing to know any login information. I also decided that I wanted the experience to be exactly as though the admin user had logged in as the other user. When the admin user is done being logged in as a band or a user, he should be able to hit the logout button and magically be logged in as his real (admin) user again, just as he was before he started.

Zend_Auth will store one identity at a time, but it allows you to use an auth_adapter to control who is logged in and when. So originally I thought it made sense to store a “masquerading” identity and a “real logged in person” identity in my adapter. What about a more complex situation where there are several levels of masquerading? What if a higher admin masquerades as a lower admin, and then as the lower admin masquerades as a user? So I decided that instead of keeping two identities, I would keep a queue of logged-in users in the auth adapter.

/**
 * Medical_Auth - an auth adapter for use with zend_auth
 *
 * The adapter maintains a queue of logged-in people. At any given time you are
 * logged in as the person on the top of the queue. If you masquerade as another
 * user, your credentials are kept in the queue, while the other user's credentials
 * are pushed to the top of the queue. When you log out (stop masquerading), your
 * credentials are now at the top of the queue. You can masquerade as many levels
 * deep as your heart desires.
 *
 */

class Medical_Auth implements Zend_Auth_Adapter_Interface {
	protected static $sess;

	// instantiation of the auth adapter stores credentials to queue
	public function __construct($email, $hashedpassword) {
		self::getSess();
		self::addCredentials($email, $hashedpassword);
	}

	protected static function getSess() {
		if (!isset(self::$sess)) {
			self::$sess = new Zend_Session_Namespace('medical.auth');
			if (!is_array(self::$sess->emails)) {
				unset(self::$sess->emails);
				unset(self::$sess->passwords);
				self::$sess->emails = array();
				self::$sess->passwords = array();
			}
		}
	}

	// push credentials to top of queue
	protected static function addCredentials($email, $hashedpassword) {
		self::getSess();
		self::$sess->emails[] = $email;
		self::$sess->passwords[] = $hashedpassword;
	}

	// attempt to log in with email and password, using zend_auth
	// -passing blank strings just uses the latest thing off the top of the queue
	// -because it instantiates a medical_auth instance, it stores new creds to queue
	protected static function forceLogin($email='', $hashedpassword='') {
		self::getSess();
		$newcreds = strlen($email);
		if (!$newcreds) {
			if (count(self::$sess->emails) == 0) {
				return new Zend_Auth_Result(Zend_Auth_Result::FAILURE_IDENTITY_NOT_FOUND);
			}
			$email = array_pop(self::$sess->emails);
			$password = array_pop(self::$sess->passwords);
			$adapter = new Medical_Auth($email, $password);
		} else {
			$adapter = new Medical_Auth($email, $hashedpassword);
		}
		$auth = Zend_Auth::getInstance();
		return $auth->authenticate($adapter);
	}

	// returns true on successful masquerade, false otherwise
	public static function masquerade($email, $hashedpassword) {
		Zend_Auth::getInstance()->clearIdentity(); //forget logged in user
		$result = self::forceLogin($email, $hashedpassword);
		if ($result->isValid()) {
			return true;
		} else {
			//revert to old login.
			self::forceLogin(); // no credentials means go to old creds
			return false;
		}
	}

	/**
	 * Performs an authentication attempt
	 *
	 * @throws Zend_Auth_Adapter_Exception If authentication cannot
	 *                                     be performed
	 * @return Zend_Auth_Result
	 */
	public function authenticate() {
		self::getSess();
		$email = self::$sess->emails[count(self::$sess->emails) - 1];
		$password = self::$sess->passwords[count(self::$sess->passwords) - 1];
		$db = Medical_Db::getInstance();
		$result = $db->fetchAssoc('select * from user where email = ?' , $email);
		$result = array_values($result); // reset keys
		$ret = NULL;
		if (!count($result)) {
			// user does not exist
			$ret = new Zend_Auth_Result(Zend_Auth_Result::FAILURE_IDENTITY_NOT_FOUND,$email);
		} else if ($result[0]['password'] != $password) {
			// wrong password
			$ret = new Zend_Auth_Result(Zend_Auth_Result::FAILURE_CREDENTIAL_INVALID,$email);
		} else {
			$ret = new Zend_Auth_Result(Zend_Auth_Result::SUCCESS, $email);
		}
		if (!$ret->isValid()) {
			// do not store invalid login credentials;
			array_pop(self::$sess->emails);
			array_pop(self::$sess->passwords);
		}
		return $ret;
	}

	public static function hashPassword($password) {
		return md5('salt' . $password);
	}

	public static function hasIdentity() {
		return Zend_Auth::getInstance()->hasIdentity();
	}

	public static function getIdentity() {
		if (!self::hasIdentity()) return false;
		return Zend_Auth::getInstance()->getIdentity();
	}

	// log out
	// if there are more credentials in the queue, log back in with them
	public static function clearIdentity() {
		self::getSess();
		Zend_Auth::getInstance()->clearIdentity();
		array_pop(self::$sess->emails);
		array_pop(self::$sess->passwords);
		$email = '';
		$password = '';
		if (count(self::$sess->emails) > 0) {
			// array_pop is the right thing because forceLogin will add it back to the queue
			$email = array_pop(self::$sess->emails);
			$password = array_pop(self::$sess->passwords);
		}
		if (!strlen($email)) {
			// nothing more to grab from the queue, so log out for real
			self::clearEverything();
			return;
		}
		$result = self::forceLogin($email, $password);
		if (!$result->isValid()) {
			// somehow the data on the queue was wrong. This is weird. Log out.
			self::clearEverything();
		}
	}

	// a real life log out. Clear everything, destroy everything, log out all the way.
	protected static function clearEverything() {
		self::getSess();
		unset(self::$sess->emails);
			unset(self::$sess->passwords);
			self::$sess->emails = array();
			self::$sess->passwords = array();
			// ran out of creds to log back in.
			Zend_Session::destroy(true);
	}
}

As you can see, there are plenty of helper methods in there for simplifying common tasks like checking to see if you’re logged in. But because I’m always using the built-in Zend_Auth mechanisms for storing the credentials, the standard things like Zend_Auth::getInstance()->getIdentity() will still work just fine. Of course, logging out should always be done through the provided Medical_Auth::clearIdentity().

As a sidenote, I have changed my Medical_Auth::hashPassword() method slightly. It does a little more than just adding a salted string and hashing in real life.

Anyway, in my authentication controller, where I process my login form, the code goes something like this:

			// check credentials
			$adapter = new Medical_Auth($form->getValue('email'),
									    Medical_Auth::hashPassword($form->getValue('password')));

			$auth = Zend_Auth::getInstance();
			$result = $auth->authenticate($adapter);
			if (!$result->isValid()) {
				// Invalid credentials
				$this->_helper->flashMessenger('Invalid credentials provided');
				$this->sess->login_form = $form;
				$this->_helper->redirector('login');
			} else {
				$this->_helper->redirector('loggedin-home','index');
			}

And to log out:

    public function logoutAction() {
		if (Medical_Auth::hasIdentity()) {
			Medical_Auth::clearIdentity();
		}
        $this->_helper->redirector('login'); // back to login page
    }

(the login page will just redirect to loggedin-home if the person is still logged in)

So you can see it’s actually really simple to do standard logging in and out. In the admin controller, when masquerading is requested, after policing rights and making sure the person logged in really is an admin, the following code puts the user into “chameleon mode”:

		if (Medical_Auth::masquerade($email, $password)) {
			$this->_helper->redirector('loggedin-home','index');
		}

This is probably obvious to most of you, but I’m still getting used to my mac.

Between config files, svn commands, ssh’ing, and tailing errors logs, I live in terminal a lot. I made my terminal window transparent with a dark background. I find that when I hit command-N to get a new terminal window, the new window is just white with black text.

The solution? Click on the terminal window that is styled the way you want, then go to Shell->Use Settings as Default.

I feel silly that I didn’t figure this out in the first 3 seconds.

Lately I’ve been looking into researching svn repositories somewhere other than my personal servers. A friend tipped me off to free svn servers out there, and after some googling I had come up with quite a few. I had a few important requirements:

1. Free to use on small / personal projects
2. Multiple user accounts possible. I often work with other developers and designers.
3. Reasonable bandwidth and space limits.
4. Private repositories available. Some of my stuff is not open source.
5. Ability to interface with a bug tracker if necessary.
6. Ability to host git repositories is a plus.

Let’s compare services I found.

I must say that Project Locker is the clear winner to me!

I find myself using the Zend Framework more and more as I work on web applications. Recently I went back to an older project which was based on Zend 1.3, an ancient relic from the distant past of over a year ago. Currently Zend is on version 1.9. I realized that I had never really upgraded my version of Zend in any project, and decided that I wanted an automated process that kept all my projects up to date.

My solution is to use svn externals. The basic idea is that when I call ’svn up’ on my library, it updates from both my repository and the external zend repository.

I decided to use the trunk, located at the following URL:

http://framework.zend.com/svn/framework/standard/trunk/

The first step is to decide where Zend will live. I’m doing this for a brand new project called “medical.” I created the following directories:

medical/library
medical/application
medical/public

Here, medical is the checked out version of my trunk.

I created a file called ~/zend_externals which contains the following definition:

Zend http://framework.zend.com/svn/framework/standard/trunk/library/Zend

The folder “Zend” will have an external repository pointing at the trunk of the Zend library.

Finally, navigate into the library directory and set up the external:

cd medical/library
svn propset svn:externals -F ~/zend_externals .

Now you must commit the new properties to your own repository and perform an svn update to pull down the latest and greatest.

svn up
svn ci -m "setting up svn:externals for zend"
svn up

It really is that simple. I’m doing this on all my projects from now on.

One really cool project I have been working on is called Zingtxt. Its purpose is managing sms messages for a company or large organization. For example, the radio station Kall700, which does AM radio coverage of sporting information and events in Utah, is using the system to track listener SMS messages.

I built the system together with a gifted designer. He did most of the front end code, and I did the processing on the SMS messages.

When a message hits the system, a php dispatcher routes the message to the appropriate module. So, on one short code (a 5 or 6-digit phone number), several different keywords can be set up to forward the messages into different channels. There can be subscriber lists, auto replies, or just generic feedback channels. Other channel types are in the works.

The feedback channel is far and away the coolest part of the project. Feedback channels can be used as a catch-all for a short code, or used in combination with a keyword. When a message is sent to a feedback channel, it automatically shows up at the top of the list of messages, with no need to refresh the page. Clicking on a message opens a new “tab” right inside the page (no reload required), which shows all the texts that have ever been received from (or sent to) that phone number within this channel. It’s a sort of thread view for that phone number.

The icons on the right are actions: reply, block, hide. Along the top of the messages area, there are options to view hidden messages, sent messages, or search for specific text in real time.

The system also has an interface for user management. It allows for many providers (companies) to exist, and each provider can have many programs. For example, a radio station company (let’s call it Tech Media) might have 5 different stations. Each station has its own program, and each program belongs to the Tech Media provider. Then, within the KROCK 99 program, there might be a subscriber list for an ongoing contest, an autoreply message for information about an upcoming concert, and a catch-all feedback channel. Users of the radio station can log in to their provider and view channels to which their managers have given them access. Tech Media managers can create new users and assign them to specific programs and even to specific channels. It is all a permission-based system.

Furthermore, administrators can masquerade in non-related programs and providers, to help solve problems and see what remote users are seeing.

Zingtxt was fun to build. It’s not finished. There’s still a lot of polishing to do, and future work will include additional channel types.

The technical stuff

I built the backend for zingtxt using php. We decided to use the Zend Framework. The frontend is (mostly) xhtml-compliant HTML, CSS, and javascript. We used jquery to simplify various tasks. On ajaxy pages (like the feedback channel), requests are made to a php controller, which returns JSON responses.

What did I learn?

It was interesting to interface with the SMS gateway. We just sent and received through web service calls, so it was pretty simple, but more importantly, it was a learning process to build something with all the limitations of SMS in mind.

I really liked using the Zend Framework for this project. I discovered, however, that doing extremely customized forms using Zend_Form can prove to be an arduous task. This would normally have been a problem, but the designer I was working with insisted on customizing beyond the normal capabilities of Zend_Form. To his credit, the site looks much nicer because of it. Unfortunately, it ended up being an awful lot of work to tweak our forms that much. I think if I had to do the project again, I might choose to use a different framework or to create an extensive wrapper for Zend_Form.

We probably spent too much time fixing minor details rather than implement further core functionality. This is something I have experienced in other projects. It can be difficult to find a middle ground between quality and quantity.

I wish I had built in more automated tests. It would have made our deploy cycle simpler and faster.

It was a lot of fun to work with a seasoned designer. I discovered quickly how to pass data in a way that made sense to him, and how best to prepare data for him to use in the views. I definitely liked using JSON for our ajax pages. Having the designer on board helped me keep a vision of the user experience, instead of delving into the technical aspects of the project too deeply to stay connected to the “layman” view of the app.

While working on an sms management system recently, I became less-than-enamored with the idea of creating several Zend_Db-related objects in various places in the application.

One solution would have been to instantiate a connection in the bootstrap, and shove it into the registry. The code to accomplish this would have looked something like:

$db = new Zend_Db_Adapter_Pdo_Mysql(array(
    'host'     => 'localhost',
    'username' => 'user',
    'password' => 'pass',
    'dbname'   => 'whatever'
));
Zend_Registry::set('db', $db);

// later, in a model or someplace:
$db = Zend_Registry::get('db');

But what if later I wanted to add new functionality to the database object? And so I decided to create a singleton to wrap the Zend_Db functionality.

I realized that there could very easily be situations where I might need connections to multiple databases simultaneously, and so having only one Db instance might not be ideal. So I decided to put an array of instances in the class, and allow the coder to specify the instance name he wished to retrieve. Furthermore I decided that these database “aliases” would correspond to a label in the config file.

// not quite a singleton, more like a wrapper for zend_db.
class Zing_Db {
    private static $instances;

    // A private constructor; prevents direct creation of object
    private function __construct($dbalias = '') {
		// pull configuration from application.ini and connect to database.
		$config = new Zend_Config_Ini(APPLICATION_PATH . '/configs/application.ini', APPLICATION_ENV);
		if (!strlen($dbalias)) {
			$params = $config->resources->db->params;
			$adapter = $config->resources->db->adapter;
		} else {
			$params = $config->$dbalias->params;
			$adapter = $config->$dbalias->adapter;
		}
		self::$instances[$dbalias] = Zend_Db::factory($adapter, $params);
	}

    // The singleton method
    public static function getInstance($dbalias = '') {
		if (!is_array(self::$instances)) self::$instances = array();
		if (!array_key_exists($dbalias, self::$instances)) {
			new Zing_Db($dbalias);
		}
		return self::$instances[$dbalias];
    }

    // Prevent cloning
    public function __clone() {
		trigger_error('Clone is not allowed.', E_USER_ERROR);
    }
}

The configuration looks something like this:

[environment]
resources.db.adapter = "mysqli"
resources.db.params.host = "192.168.7.7"
resources.db.params.username = "fakeuser"
resources.db.params.password = "fakepass"
resources.db.params.dbname = "fakedbname"
resources.db.isDefaultTableAdapter = true

anotherdb.adapter = "mysqli"
anotherdb.params.host = "192.168.7.7"
anotherdb.params.username = "fakeagain"
anotherdb.params.password = "fakepass"
anotherdb.params.dbname = "someotherdbname"
anotherdb.isDefaultTableAdapter = false

Of course, my usernames, ips, passwords, and database names are all fake in the above example. The variable APPLICATION_ENV in the php code correlates with the environment name in the config file, and the available db names are “anotherdb” or “”, which uses the default database. I am using Zend_Application, so resources.db is the default database.

The code to use a database is now something like:

$db = Zing_Db::getInstance();

    - or -

$count = Zing_Db::getInstance('anotherdb')->fetchOne('select count from user');

Suppose you are a few months into the project and you’d like to take a look at all the queries being run and how long they take. You could build this right in to the database class.

	public static function doProfiling($asHtml = true) {
		$ret = '';
		$sp = $asHtml ? ' ' : ' ';
		foreach (self::$instances as $dbalias => $db) {
			$p = $db->getProfiler();
			$ret .= "Profile for database: $dbalias <br/>";
			$ret .= 'Total queries: ' . $p->getTotalNumQueries() . '<br/>';
			$ret .= 'Seconds spent: ' . $p->getTotalElapsedSecs() . '<br/>';
			$queries = $p->getQueryProfiles();
			$ret .= '<br/>--------------------<br/>Query information: <br/>';
			if (is_array($queries)) {
				foreach ($queries as $q) {
					$ret .= $q->getQuery() . '<br/>';
					$params = $q->getQueryParams();
					if (count($params)) {
						foreach ($params as $k => $v) {
							ob_start();var_dump($v);$v = ob_get_clean();
							$ret .= "$sp$sp$k: $v<br/>";
						}
					}

					$ret .= $q->getElapsedSecs() . ' seconds.<br/>';
					$ret .= '--------------------<br/>';
				}
			}
		}
		if (!$asHtml) {
			$ret = br2nl($ret);
		}
		return $ret;
	}

I admit that appending output onto a return variable is kind of ugly, but in this case (non-user-facing code which a designer will never touch), it seemed appropriate.

How do you use the profiler? Create another configuration value in your ini file:

resources.db.params.profiler = true
anotherdb.params.profiler = false

Profiler is another parameter you can pass to Zend_Db::factory, so we don’t have to change the Zing_Db code at all.

Where you want to see the output (maybe near the bottom of a layout script?) just throw in something like this:

echo Zing_Db::doProfiling();

This post is just a link to John Resig’s “Learning Advanced JavaScript” – one of the most concise and awesome javascript tutorials I have ever seen.