SuccessWhale is Terrifying: VPS Edition

Just under two years ago, my Suc­cess­Whale Twit­ter client was gain­ing new users at a steady rate and, as I noticed with alarm, was about to blow through my then-limited band­width allowance.

I’ve since relo­cated all my web stuff to Dreamhost, tak­ing advan­tage of their unlim­ited band­width offer­ing to plow through 10 GB and more a month. But now I’m com­ing up against the last remain­ing limit of my shared host­ing — mem­ory usage.

Both West­min­ster Hub­ble, which con­stantly crawls MPs’ social net­works and RSS feeds, and an increas­ingly com­plex Suc­cess­Whale, churn through a ton of mem­ory. I don’t have a nice scary graph for this one, but at peak times, I’d esti­mate that my web server kills over half my PHP processes due to excess mem­ory use. That means Only Dream­ing basi­cally goes down, while Suc­cess­Whale throws errors around if it even loads at all.

It looks like I’m left tak­ing the expen­sive plunge of mov­ing my host­ing to a VPS rather than a shared solu­tion, which is a jump I’m ner­vous to make, espe­cially since none of my web prop­er­ties make me any money. Most wor­ry­ing of all is that VPS prices tend to vary by avail­able mem­ory, and I don’t actu­ally know how much mem­ory all my stuff would take up if it were allowed free rein. And nor do I have any way of find­ing out, bar jump­ing ship to a VPS and tak­ing advan­tage of free trial weeks.

So, dear lazy­web, do you have any expe­ri­ence with this sort of thing? And can any­one rec­c­om­mend a good (cheap!) VPS host that ful­fils the fol­low­ing criteria:

  • LAMP stack with “P” being both PHP and Python (or *BSD instead of Linux)
  • Full shell access
  • Unlim­ited (or at least 100 GB) bandwidth
  • Unlim­ited (or at least 10 GB) disk space
  • At least 20 MySQL databases
  • IMAP mail­boxes & mail forwarding

I’ve been rec­om­mended lin­ode by a friend which seems great for tin­ker­ing, though the price scales up rapidly with RAM use and I’m not sure I want to deal with the has­sle of set­ting up Apache, MySQL etc. by myself. And there’s Dreamhost’s own offer­ing, which would be vir­tu­ally zero-hassle to switch to, but prob­a­bly isn’t the cheap­est around.

So, cit­i­zens of the inter­web, I seek your advice!

Cobb’s Quay Weather

Cobb’s Quay in Poole Har­bour has a crappy Flash-only front-end to its weather sta­tion. My “Cobb’s Quay Weather” page grabs the data files behind the Flash, and dis­plays the impor­tant bits with min­i­mal for­mat­ting, e.g. for mobile browsers.

Made for a friend of mine who couldn’t get the Flash inter­face to work from his phone.

Note: Tem­per­a­ture read­ings seem to be bro­ken at the moment (April 2011). This is an issue with the source data, not my script.

You can:

The Marvellator

“The Mar­vel­la­tor” is a sim­ple PHP script that gen­er­ates ran­dom, bizarre or pos­si­bly wor­ry­ing comic book titles, usu­ally in the style of “The Amaz­ing Spi­der­man”, “Incred­i­ble X-Men” etc. I think the most hor­ri­fy­ing prospect it’s gen­er­ated for me is “The Adven­tures of Tiny Rorschach” — even more hor­ri­fy­ing because it actu­ally exists. (Thanks Frankie.)

Nat­u­rally, this has noth­ing to do with Mar­vel Comics in any way (please don’t sue me).

You can:

Announcing: Daily Promise!

After a cou­ple of weeks of devel­op­ment — doc­u­mented here, here and here — I think I’m ready to call Daily Promise ver­sion 1.0.

It’s a site that helps you keep track of your promises day-to-day, giv­ing you a pretty dis­play of which promises you’ve kept when, and let­ting you com­pete against your Twitter-using friends to be the best at keep­ing your daily promises!

For now, you can find Daily Promise at http://dp.onlydreaming.net.

I’ll make the same deal as I made with Dynamic Democ­racy, but dou­bling the num­ber so that I can be more sure of it tak­ing off, and that’s the following:

At the moment the site does every­thing I want it to do, and it’s hosted on a sub­do­main of my main web­site, which I have no prob­lem with. What I would like to do is give it its own domain, and start imple­ment­ing fea­ture requests that peo­ple send in. So that I don’t end up spend­ing money on some­thing that’s going to die off quickly, the deal is this: When it gets 20 active users, it gets a domain and some TLC. If it doesn’t make it to that point, it stays like it is.

So if you’d like to help me make some­thing of this site, please start using it, and show it to any of your Twitter-using friends who might need a lit­tle help get­ting healthy, keep­ing fit or any other goal that Daily Promise can help them with!

Daily Promise: Avatars Everywhere!

After a cou­ple of days and one fran­tic family-free morn­ing, Daily Promise is get­ting near com­ple­tion. Here’s what’s new since last time.

(This is post num­ber 3 in my series on the devel­op­ment of Daily Promise. The oth­ers are here: Design Sketches, Com­ing Together.)

Friends Page

Daily Promise: FriendsHere’s the Friends page — again, almost no devi­a­tion from the orig­i­nal design sketch. The friends page pulls in the list of peo­ple that you fol­low on Twit­ter, matches it up against Daily Promise’s user list, and if any match, they’re your Daily Promise friends! They’re sim­ply dis­played in alpha­bet­i­cal order, along with a sum­mary of their per­for­mance. Invis­i­ble users (see later) don’t appear, even to their friends.

Nicer User Pages

Daily Promise: User PageClick­ing on one of your friends takes you through to their ‘view’ page (minus any edit­ing func­tion­al­ity). It also shows you their Twit­ter bio, and how long they’ve been using Daily Promise.

Top Users Widget

Daily Promise: Top Users WidgetThere’s now a “top users this week” wid­get on the home page, show­ing the per­for­mance of the top 5 users. This resets at mid­night on Mon­day morning.

Spam your Friends!

Daily Promise: Tweet BoxTwit­ter inte­gra­tion now includes boxes sug­gest­ing Tweets you might like to make after each sig­nif­i­cant activ­ity. Just as promised in the “How does it work” graphic, Daily Promise never posts to your Twit­ter account with­out you delib­er­ately click­ing a “Tweet” but­ton each and every time. Do no evil™!

Behind the Scenes

A lot of other stuff has changed in the last few days that isn’t imme­di­ately obvi­ous to users:

  • Authen­ti­ca­tion fixed — users using the alter­na­tive login weren’t able to do Twit­ter things. That’s sorted.
  • Account vis­i­bil­ity — your account can now be set to invis­i­ble, mean­ing it won’t appear any­where — top users, friends lists, etc. New accounts are given a prompt to set their vis­i­bil­ity before start­ing to add promises.
  • Account dele­tion sim­pli­fied — you now only have one, nuclear, option for account dele­tion. It erases all traces of you hav­ing used the site. Do no evil™! :)
  • Removed promises no longer shown in the his­tory table — ‘cos no-one likes to be reminded.
  • Fill in data for yes­ter­day — when cre­at­ing a promise, users can opt to enter data for yes­ter­day, giv­ing them some­thing to fill in straight away.
  • His­tory table scrolls — nar­row dis­plays can’t fit the whole his­tory table in, so now it scrolls (in rea­son­ably mod­ern browsers).
  • Time zones imple­mented — we pull the time­zone you have set in Twit­ter, so Daily Promise will roll over to a new day at your local midnight.
  • Crontas­tic! — we now update stats and things from an hourly timed cron, to avoid extra load­ing on user-requested pages.

Next Steps

This all brings me to the slightly wor­ry­ing con­clu­sion that Daily Promise is damn near fin­ished. So, where do we go from here? I’ll have a few more days of bug-fixing and imple­ment­ing fea­tures that peo­ple request, and then it’s dif­fi­cult deci­sion time:

This has been a fun project for the last week or so — does it deserve a domain and adver­tis­ing, or shall I let it qui­etly die?

Daily Promise: Coming Together

Despite the lack of response to my ear­lier post, in which I floated my design con­cepts for “Daily Promise”, bore­dom won out in the end and I started cod­ing anyway.

It’s now com­ing together, and all bar the Twitter-integrated social aspects are largely com­plete. Here’s how it’s developed:

Home Page

Daily Promise: HomeThe social side — top users, etc. — still isn’t imple­mented, but there’s a reasonable-looking home­page in there. The main body is taken up with a short descrip­tion and a big graphic explain­ing how the site works. Side-bar wid­gets pro­vide the Twit­ter login and alter­na­tive login (bypass­ing twitter.com). The site now has a proper name, Daily Promise, and with it a logo and style that is reflected throughout.

Set Up Goals (“Manage”)

Daily Promise: ManageThe “Man­age” page has remained almost exactly faith­ful to the design. New promises can be cre­ated, old ones deac­ti­vated and deac­ti­vated ones can be acti­vated again. A Tweet box appears for the user to announce their new promise, if desired.

Daily Per­for­mance (“Enter”)

Daily Promise: EnterAgain, there’s not a lot of dif­fer­ence here between the design and the real­ity. Each promise has a yes/no choice, and after com­plet­ing a day’s entries, Tweet boxes appear for the user to let their friends know about their suc­cesses and fail­ures. “Win­ning streaks” aren’t yet implemented.

Per­for­mance Log (“View”)

Daily Promise: ViewThere’s no abil­ity to scroll through your his­tory yet, but the default dis­play shows 4 weeks (which scroll if nec­es­sary). Just as in the design draw­ings, the his­tory table is fol­lowed by a text sum­mary of how the user is doing.

The “View” page also, with a few addi­tions, becomes a user’s pro­file page, which is acces­si­ble to other users.

Con­fig­u­ra­tion

Here you can set your pass­word for the alter­na­tive login, and delete your account. It’s exactly as dull as it sounds.

Friends

That’s my big job for the next few days! It doesn’t exist yet, but it’s now my top priority.

Daily Promise: Design Sketches

Cur­rent flavour of the month of some of the geek crowd, “Health Month”, is a social net­work of sorts on which users com­pete to achieve cer­tain health-related goals. Each month, each mem­ber sets a num­ber of goals for them­selves to achieve. Its core mechanic is health points — you start with 10, lose one every time you fail to meet a goal, and play­ers who per­form well can heal you.

I’m enjoy­ing my use of the site with three goals this month, but I’d like to step it up and set lots. Unfor­tu­nately, hav­ing more than three goals costs money. (Not that I think the site’s own­ers don’t have a right to charge, but it can be a deter­rent to users such as myself.) It also cur­rently only allows two “cus­tom” rules per month — beyond that, you have to stick with the pre-defined ones.

Another social health site is Tweet What You Eat, on which users tweet their food intake and have the site, or the com­mu­nity, cal­cu­late sta­tis­tics such as their calo­rie intake.

Over my lunch hour, I’ve come up with some sketches for a site that sits some­where between the two. It takes Health Month’s goals mechanic, opens it up and removes some of the social aspects that in my opin­ion Health Month doesn’t imple­ment all that well. It also drifts closer to Tweet What You Eat, in that rather than being its own ser­vice it pig­gy­backs of Twit­ter for its social side.

At the moment this is just a fun con­cept I’m toy­ing with — I don’t really have the time to make it at the moment, I doubt the space between Health Month and Tweet What You Eat is wide enough to make a new site pop­u­lar, and I feel a lit­tle guilty about thank­ing Health Month for the enjoy­ment I’ve had by becom­ing its competitor.

In the notes below it’s dubbed healthi.ly, though as that domain is parked, it’s come to be known as “Daily Promise” instead.

Home Page

Daily Promise Home PageThe home page would largely be a “log in / reg­is­ter” affair, pos­si­bly also show­cas­ing suc­cess­ful and pop­u­lar users in a side-bar (not shown). Big ban­ner text explains the rough con­cept, with a “read more” link to a full “About” page. On the reg­is­tra­tion side, we make it clear exactly what Daily Promise does and doesn’t do with access to your Twit­ter account.

Set Up Goals

Daily Promise Goals PageThe main setup page is where you set your goals. Users can set any (rea­son­able) num­ber of goals, they can drop and res­ur­rect old ones, and add new ones, at any time. Per­for­mance against all the goals is tracked and vis­i­ble on this page. Adding new goals and drop­ping old ones can be tweeted, but as with every tweet oppor­tu­nity, the user is pre­sented with an @Anywhere box that they can freely edit and can choose not to tweet as eas­ily as they can choose to tweet. The tweet links to the list of goals on their profile.

Daily Per­for­mance

Daily Promise Daily Performance PageOnce goals are set, the user logs in each day (and can fill in past gaps) with whether or not they have met each goal. Each day’s entry presents some brief sta­tis­tics, and you get more stats on the week after fill­ing in Sunday’s per­for­mance. Very good or very bad per­for­mance sug­gests a Tweet that a user might like to make. The tweet links to their per­for­mance log on their profile.

Per­for­mance Log

Daily Promise Performance LogThis is a user’s main screen. It dis­plays a chart of passes and fails for the last month or so as green (pass), red (fail) or grey (goal not active) squares. Below the chart, more detailed stats are pre­sented, as well as an encour­ag­ing text sum­mary of how the user is doing.

Set­tings

Most of the core set­tings such as user­name, dis­play name, avatar and bio are han­dled by Twit­ter. Daily Promise’s set­tings prob­a­bly boil down to pri­vacy (stop me being search­able, delete my account, etc.) and remov­ing annoy­ances (always tweet on con­di­tion x, never tweet on con­di­tion y, etc. — all of which have an “ask me” set­ting by default).

Friends

Daily Promise Friends PageThe user’s “fol­low­ing” list from Twit­ter is used to gen­er­ate their list of Daily Promise friends. Avatars, user­names and Daily Promise per­for­mance sum­maries are dis­played here. Click­ing through to a user’s pro­file shows the “per­for­mance log” page, topped with name / avatar / bio / etc.


So, and inter­est­ing idea, or an appalling one? Would you use this? Should I get off my arse and code? Should I have fin­ished the last six things I started before pro­to­typ­ing some­thing new? Your thoughts are, as always, appreciated.

A Farewell to Marmablues

May 1998, half a life­time ago. It was my 13th birth­day, and my par­ents — no doubt annoyed by four years of me mess­ing with the fam­ily com­puter — bought me my own. It had a 333MHz proces­sor, 32 glo­ri­ous megabytes of RAM, and most excit­ing of all, a 56k dial-up modem.

With Microsoft Word as my co-pilot and under the ever-watchful phone-bill-monitoring eyes of my par­ents, I dis­cov­ered the delights of own­ing my own web­site. It had it all, oh yes. Giant back­ground images, a dif­fer­ent one for each page. Ani­mated GIFs. Back­ground MIDIs. Frames, <blink> and <marquee>. Web rings to click through, and Tripod’s ban­ner ads inserted at the top of every page. It was called The Mad Marmablue Web Por­tal, and it was exactly as hor­ren­dous as you are imagining.

A few years later, a chronic lack of smallprint-reading led me to buy it a ‘free’ domain name, only to receive a scary-looking invoice a month later. In the end my par­ents sent the domain reseller a let­ter explain­ing that I was a dumb-ass kid who shouldn’t be trusted on the inter­net, and that was that. But at the age of sev­en­teen, in pos­ses­sion of a Switch debit card, I found a web host who would set me up with a domain and 100MB of space for £20 a year. marmablue.co.uk was born.

Today, it died.

I shouldn’t feel sen­ti­men­tal about jet­ti­son­ing an old unused domain, par­tic­u­larly not one that harks back to the late-90s ani­mated GIF hor­ror of the Mad Marmablue Web Por­tal. But it was a part of my youth, the place where for the first time I could put some­thing and any­one could see it. It was where I took my first steps with HTML — by rip­ping off other web­sites, nat­u­rally — and in time, it was where I first learned JavaScript, PHP and SQL too.

I will miss it. But if I sit very still, and very qui­etly, I can still hear that hor­ri­ble 8-bit MIDI ren­di­tion of the Robo­Cop theme tune. So maybe I won’t miss it all that much.

a thousand words: Finishing Touches

The vast major­ity of user-reported bugs and requested fea­tures on “a thou­sand words” have now been sorted out. As requested by my co-conspirator Eric, we now have an ‘adult con­tent’ fil­ter based on a date of birth field in users’ pro­files, and a ‘report’ but­ton to bring prob­lem­atic sto­ries and pic­tures to the atten­tion of the mod­er­a­tors. There’s also a DeviantArt-style “request cri­tique” option to let users know what kind of com­ments you’re look­ing for.

Time­stamps have been fixed, “no stars yet” rat­ings intro­duced, and text field poli­cies such as “mustn’t be empty” have been added across the site. A few ren­der­ing issues in IE have been sorted out, so it now looks much the same across all platforms.

The biggest change is unfor­tu­nately some­thing most of you will never see — the mod­er­a­tor con­sole. Pic­ture sub­mis­sions and reported stories/pictures now sit in queues that can be dealt with by mod­er­a­tors. An item enter­ing a queue trig­gers an e-mail to all mods, who are invited to review it and make changes as appro­pri­ate. Once changes are made, the affected users are then e-mailed to let them know what hap­pened (and in the case of reported items, to give them a chance to chal­lenge it).

There’s one major fea­ture request that’s not yet been imple­mented: file uploads. Once in the sys­tem this would allow users to sub­mit pic­tures from their hard dri­ves rather than from the web by URL, and would allow mod­er­a­tors to copy URL-linked pic­tures to the site to avoid hotlink­ing. (At present we don’t hotlink, but we do there­fore have to copy pic­tures to the site man­u­ally using FTP.) It could also allow users to use a non–Gra­vatar pic­ture for their profile.

Depend­ing on how things go, that may or may not be ready by tomor­row night. On Sat­ur­day morn­ing I jet off to sunny Saudi Ara­bia, so any changes not made by then are going to remain unmade for a while. From that point it’s in Eric’s capa­ble hands as to whether she wants to release the site or not. Even if the site does advance to release sta­tus, I’m still tak­ing bug reports (they’ll sit in my inbox until I get back), so keep on let­ting me know what’s bro­ken and what you’d like to see added!

a thousand words: Alpha, Beta

“a thou­sand words” has now reached a stage where every fea­ture that I give a damn about is imple­mented. Thus, we’re open­ing it up to a lim­ited beta test to iron out the wrin­kles and get a list of any fea­tures poten­tial users would like to see us launch with. If you’re bored or sim­ply have a love of break­ing other people’s shit, head along to http://athousandwords.org.uk and see what hell you can raise. As the Big Red Box Text warns you, really don’t sub­mit any work of fic­tion you care about, just in case some kind soul finds an SQL injec­tion vul­ner­a­bil­ity and trashes the database.

Since last time I bored the hell out of you all, vot­ing and com­ment­ing has been imple­mented, reg­is­tra­tion has been fixed, fil­ter­ing HTML tags from sub­mis­sions has been added, as has a word count and the pic­ture selec­tor on story sub­mis­sion. There’s been a bunch of behind-the-scenes tweaks to improve secu­rity too.

The one fea­ture that Eric def­i­nitely wants is a way to mark sto­ries accord­ing to their con­tent. We could do this in sev­eral ways — I would pre­fer, if any­thing, to just have a “not for kids” option on each post and a Date of Birth field asso­ci­ated with user accounts, so we can hide sto­ries as required. Other options include a range of rat­ings (U, PG, 12, 15, 18…) or tags for cer­tain con­tent (vio­lence, sex, lan­guage) so peo­ple can avoid what­ever they’re picky about.

This prob­a­bly ought to come with a Report but­ton so that users can report incor­rectly rated sto­ries, and I would add a sim­i­lar fea­ture to report pic­tures. (Pic­ture sub­mis­sions are mod­er­ated, so Goatse isn’t going to make it through any­way, but the mod team might miss sub­tler things like licenc­ing terms and copy­right infringement.)

At that point, all that’s left on my list is the admin inter­face and any­thing that users sug­gest dur­ing this beta. Hope­fully we’ll be ready to launch by the time I depart for sandier shores at the end of the week!