Institutional Knowledge

Metrics are a funny thing. We have 4 main ways of looking at the portal. We have concurrent users, which we measure at 5 minute intervals. We track logins and unique users on a daily basis. We have Google Analytics. Last but not least, we also track how many people click through to PeopleSoft through our CAS access logs. For instance, here is what the first day of spring term looked like in terms of concurrent users on a hourly basis.

Yeah, it was hopping. You can even seen the network outage we had late that night (yeah, that was fun). This is just a snapshot though. It just tells us that, in general, it was really busy.

Our login stats also show that Monday was indeed very, very busy with 31,984 logins made by 13,412 users. A good number to have, but it doesn’t say much. We don’t currently run an analysis to get a real logins per user average.

Google Analytics shows us where people went in the portal but not where they went when they were leaving the portal.

We know there are three main destinations: PeopleSoft, WebCT/Blackboard Vista, and webmail. Because of the way we send users to PeopleSoft from the portal, we have good numbers — although they are not tied to unique users.

Clearly the first day of the term (the 28th) shows a lot of activity. We rack up +70k clicks for only 13,412 users and 31,984 logins. The performance of the PeopleSoft system suffered a little that day, which is what really prompted us to start trying to tie all these numbers together.

While nothing is definitive, it seems that the performance problems were due to a number of issues — as you would expect in an enterprise system like PeopleSoft. With ~32k logins and some poor statistical assumptions you get just over 2 clicks into PeopleSoft per login and users logging in at least twice during the day.

The questions that these numbers can’t answer for us are many. Is the system usable? Are usability problems leading people to login more times than they would in a perfect world? Are students needing to login frequently on the first day to find or adjust their schedule? All of these will remain “unknown unknows” until we actually talk to students.

What do we know from these numbers? The first day of school is busy.