Monthly Archives: February 2010

You are browsing the site archives by month.

Life changes you, things learned.

It’s almost been 2 years since i moved to california and its been a rough adventurous ride, and I don’t regret a moment of it. However, when I look back it’s odd to see just how different I am today.

Since sundays are for reflection, here are a few bullet points.

  • • living in milwaukee i felt very connected to the community, i knew so many people by name that everywhere i went was like an episode of cheers.
  • • i used to drink and eat a lot more, today i eat 2/3rds to 1/2 as much food. it’s so warm in california you really don’t need much.
  • • winter seems like someplace you visit, rather than live through.
  • • i used to feel tech deprived, now it’s easily accessible and to some degree less exotic
  • • i understand better how some ideas are good ones and others are a waste of time
  • • i now know i own too much stuff, too much ownership is a burden; you end up consuming more resources just keeping it and not appreciating it.
  • • women like geeks and entrepreneurs here, i feel sexy here
  • • hackathons used to be parties, now they seem like work
  • • i miss bucketworks, but….. have found noisebridge, and techshop, but don’t go enough.
  • • owning cars seem silly, but i still love my 350z, but now i don’t have a venue to enjoy it. i may just sell it and rent a sports car for a day, even at 500 dollars every few months, it’s cheaper, than paying for insurance and parking.
  • • in milwaukee, geeks were most easily found at goth industrial clubs, but out here, they are everywhere.
  • • you live in a city, not in an apartment, neighborhoods are probably more important than your home.
  • • being able to bike everywhere RULES.
  • • public transit is better than no transit, but it still sucks, its a tragedy of the commons.
  • • some graffiti is good, some is bad.
  • • political parties are not relevant.
  • • some people are so angry about somethings, that they will never listen about other things
  • • what its like to have a lot of friends that are female, what it’s like to have a female room mate; the secret is it’s really not that different, except your guy friends probably are more likely to drink your beer and watch the same tav shows.
  • • on average women think about sex way more than I would have thought 2 years ago.
  • • women may think differently, but they think about the same things. ( ok, maybe not all of the same things, or maybe just not at the same level of detail )
  • • the goal of any good conversation is to derive respect of and for someone regardless of weather you agree, find enjoyment in their fellowship, however not all people are respectable, and so don’t spend your time with them
  • • no matter how much you may want others to be independent and self capable, some people just want to follow, or lead. people may not be built for an egalitarian maximum.
  • • any good piece of art, creates a deeper connection between people. it should make good conversation easy. ( see above )

Improving PHPBB2 and MySQL performance

For the last few days I have been trying to track down why Unity’s PHPBB 2 Forum (http://forum.unity3d.com) was so slow. Page rendering times were taking between 2 and 10 seconds and for me this was just unacceptable.

A proper website should be able to render a majority of the page in less than 250ms, and delivering most of the content in less than 50ms is ideal. Sites like Amazon, Yahoo, and Google have studied the effects of response time vs. features, and have found that response time is often times more important. Greg Linden pulls together a few sources on the topic in his blog.

So, 10 seconds was just unacceptable, and I was bound to find a solution. I started with the low hanging fruit, and installed PHP’s APC cache which is an OpCode Cache that stores compiled versions of the PHP code to reuse on the next request. There is evidence out there to suggest that not only do opcode caches reduce CPU requirements, but they also reduce your memory load. I saw a few sites that claim about a 3-4x performance increase on the CPU and about a 25% reduction in memory usage.

The next step, as the forums were still slow, was to start looking into our MySQL usage. Did the server have enough memory, were the MySQL caches and buffers large enough, or were searches and queries getting pruned to make room for more queries? After using MySQL’s GUI tools through an SSH Tunnel I was able to see that the server had 64MB of query cache, and only 90% of it was being used, so the caches were good.

Next was to look for slow queries, however a slow query or two would be hard to fix considering PHPBB was building the queries as part of the application and should have been tested in advance. If a slow query was the problem, it probably was not part of the application design, but rather an indication of something else that was going wrong. We did notice a few slow queries, but nothing ridiculous. However, what we did notice was that a number of queries were running slow because other ones where holding a lock for a long period. Hmmm?

Next we started looking at the problem from a system level using apps like ‘top’, ‘dstat’, ‘mytop’, etc… From here we found that disk IO was over working. Hmmm… but why?

I later found out that we were serving large files in our forum, however it did not seem like our forum was being used as a hot linking service, and I would suspect that PHPBB has some defense from that, but going forwards this is a concern of mine.

I finally found a post on the web talking about MySQL CPU spikes, and it was related to a corrupted table index. http://forums.fedoraforum.org/showthread.php?t=232008

After reading the forum for a bit, I thought, well there seems to be a few low risk commands I can run to look into our table integrity. Here is the full command list http://dev.mysql.com/doc/refman/5.0/en/mysqlcheck.html

First I ran:

mysqlcheck -A -F -u [username] –p

This reported several table closed improperly errors, and took about 2-6 mins to run without taking down the forums.

Next I ran:

mysqlcheck -A -q -r -u [username] –p

This is basically the fastest repair option, and would only fix minor corruption. It took about 5-10 min to run.

The result was that our MySQL usage dropped from 100-300% to 3-6% and page response times being about 200-500ms. I have a few other things I would like to try, from using our CDN with the forums to DB optimizations, to other server upgrades.

Over the weekend, I think I am going to run:

mysqlcheck --all-databases  --optimize -u [username] –p

Which should be more thorough, but might take an hour. This will defrag the table and resolve some indexing errors.

Open Loft privacy divider design.

I finally have a new place, and it’s an open concept loft in San Francisco’s SOMA district. It’s got everything I have ever wanted in a place, but it’s got one problem, to afford it I had to take on a good friend as a roommate. I love having roommates in a traditional space, but having a roommate in an open concept loft could, well ,,,.. get bothersome in the future.

I am currently exploring and brainstorming concepts to create privacy. However, building a full height wall, although a feasible engineering solution, is not what I am looking for. We are looking for something that keeps the loft as open as possible, while allowing us to define private spaces. We have thought of the following options, but what ideas might you have?

  • using plexiglass panels
  • using triangular plexiglass panels and create an uneven surface
  • using plexiglass with a negative pressure interior ( vacuum )
  • using glass
  • using foam
  • hanging rugs
  • hanging curtains
  • using bloxes
  • using a rubber skin around a substructure/ skeleton
  • creating a plexiglass substructure
  • creating a wooden substructure
  • creating a cardboard substructure
  • sucking out the air from with in the rubber encased structure
  • doing something organic looking
  • doing something modern
  • using small bright light sources like LEDs to trick the eye of an on looker to contract and thus be unable to see passed the array of LEDs, one might call this a ‘light screen’
  • use a light projector to overpower any refracting light in the room, thus creating optical privacy in one direction.
  • using some sort of blinds

I will try to update this post with examples of interesting design at some point.  Maybe you can comment on a few.