Wednesday, May 14, 2008

Twitter's Potential Monetization Path: Marketing via the Antithesis of SPAM...uh, MAPS?

If you've living in the murky, smog-like haze of a next-gen web, social networking, and mobile networking reality - you know about twitter. For those of you that don't, twitter is a short messaging/party-line system where you can broadcast up to 140 characters at a pop to your friends and family, oh....and anyone else on Planet Earth who cares to listen in.

Despite well-publicized issues with Ruby as the root of their platform scalability problems, twitter is -by any measure - a runaway success:

  • Officially a year old as of the last SouthxSouthWest conference, twitter experienced user community growth from 1M users to 3M users in the last 6 months. (Approximate, since no one knows the exact figures.)
  • A plethora of twitter applications have sprung up (I just wanted to say "plethora") with more coming every day.
  • Twitter is gaining mind share in the non-twitter community, with twitter stories popping up on CNN, the New York Times, and other major news outlets. (Sure, they get it wrong 3/4ths of the time, but, hey - they're trying.)
  • Twitter competitors Jaiku and Pownce have been largely ignored by the media, and have underwhelming user numbers, as far as anyone can tell.
  • Websites are including "Twitter Alerts" on their main pages next to the RSS feeds so that users can be sent a tweet when a new item appears.
  • Twitter received $5.4M in funding in 2007, and another $15-$20M earlier this spring.
Yet, in spite of all this traction Twitter has not found a way to monetize:
  • Any broadcasted marketing message on Twitter is seen as SPAM, and - being an opt-in system anyway, who is going to see it except people who follow the SPAMmers? (Well, alright, to be fair - I do follow some "SPAMmers," such as Blu_ray, because it tweets information to me that I want to know.)
  • The Twitter platform could be used to attach advertisements to twitter messages, but with only 140 characters to play with that leaves no room in the payload for any meaningful ad message.
  • The twitter.com website really only exists as a posting portal, and due to twitters excellent public web services API, people are using sexier twitter clients - leaving not too many people go to twitter.com anymore.
  • The sexier twitter clients could be ad supported, but the monetization would go to the client development houses, rather than to twitter.
OK, now what?

Well, twitter is definitely experimenting with ways of getting dollars out of eyeballs. Part of the twitter web services API includes calls to allow developers to tap into the constant stream of public twitter traffic and take action. (Check out the most-excellent Twistori, as one such experimental visualization application.) Over the last few months I've noticed an interesting phenomenon: several followers of mine (like Tripixdesign) are not people, but companies.

These companies don't post anything with their twitter accounts, rather they just lurk in my twitter stream...listening to what I post. How did they find my twitter stream? With access to the public twitter feed, that part is easy: maybe I posted the tweet "I hate Comcast," and suddenly I find myself being followed by comcastcares . (Yeah, that really exists.) I believe what we are seeing with these lurkers is the birth of a different kind of marketing effort... not traditional SPAM, but some sort of Black-Spiderman/Bizarro-World SPAM, which I'll call MAPS.

On the surface, this might sound similar to a move made by Google's Gmail. Late last year, Gmail added a feature that I would have thought would have made a far bigger farting noise in the public forums than it did: targeted advertising based off of content in your email. Yeah, Gmail "reads" your email, looking for keywords that it uses for targeting advertising in the pane surrounding your messages.

Google's Gmail doesn't quite fit the MAPS model - Gmail places the user into an apriori contract: you are using the Google mail service without fees, and by doing so you implicitly agree to certain conditions: placing your personal email on their servers, for instance - and, if you were to actually read their ridiculously broad and open-ended EULA, you would realize that you allowed them certain rights - such as the right to pull keywords out of your email and use that information for ad targeting. This is a form of a forced-feedback loop: marketing information is not pushed back to the user after the targeting information is pulled from the same user. No input, no targeting.

In the MAPS model, on the other hand, information is simply collected from the user, and...never seen directly by that user. The information is collected and aggregated into a targeting database unencumbered by any sort of direct marketing connection. With a couple of simple web service calls, a MAPS data collection system can:
  1. read from your twitter stream
  2. pull out keywords
  3. compare it against keywords pulled from your followers
  4. draw a conclusion about your potential path through both the real world and the web world
By going through those steps, marketers can determine where people like you (not necessarily you, however) are most likely to spend their time over the next hour, 12 hours, or 24 hours. The more you tweet, the more you tell them what you are like, dislike, plan on doing, concerts you are going to, bars you are at, trips you are taking, and people you are communicating with in the real and virtual worlds.

So - that's how other agencies, ad companies and marketers are potentially using twitter to capitalize on twitter's public twitter stream and the twitterati themselves, but what about twitter.com itself? Do it better, twitter: you received between $20M and $25M in funding, and you don't have a large staff - after operational and staff costs you probably have between $15M and $20M left. (Unless you spend more on those parties of yours than I thought.) So, with all the cash: spend your way to monetization by creating a MAPS engine for which you can charge platform fees. Do this in three steps:
  • Get the hell off of Ruby. It's clearly not doing you any favors. (Thanks for the 2 hour outage yesterday, btw.)
  • Hire analysts and statisticians to create an advanced targeting, forecasting and inference engine that uses your database of tweets to forecast where masses of eyeballs might turn to for upcoming events, and return results to agencies and marketers based off of web services queries for a fee. Marketers and advertisers could use such a tool for both checking the results of a promotion ("How'd that price drop on HP laptops do?") and forecasting what people might be interested in ("Where is the best bar in LA to promote our new light beer this weekend?")
  • Change your EULA so twitterers know what you are doing.
Powerful stuff, kids - for the first time in history, MAPS allows advertisers a way to accurately predict where whole packs of people are spending their time, energy and - most importantly - money, and they can use that information to more accurately place their billboards, hold their promotions, and sell their products.

...if that is what they are doing. I'm sure they aren't...those lurkers are probably just...listening....watching...waiting. Uh, wait that's far creeper than MAPS marketing.

Sunday, May 11, 2008

Drive Swap: Tearing open the Trusty Sony VAIO SZ370P

OK - Taking a brief break from pontification to discuss a little computer brain surgery.

Anyone who's been reading my nonsense for the past year knows that my workhorse laptop in a Sony VAIO SZ370P - a great machine that was purchased just before the Vista releases. In late Spring of '07 I upgraded the little bastard to Vista - and it ran beautifully.

Well, the SZ series from VAIO is now up to series 9 (SZ9xx), and includes faster processors, more memory and faster drives. Although I have my traditional Uber Gadget Lust for the newer SZ's, I don't want to shell out the coin for a new SZ until Sony throws a Blu-Ray drive in there... in the meantime, the 100Gig, 5400rpm drive that came with the SZ370 is starting to feel a little cramped and slow... a quick run down the Information Super Highway (tm) shows that 200Gig, 7200 rpm Seagate drives (Seagate Momentus 7200 2, or the ST9200420AS) are going for about $130 a pop.

No brainer, time to crack open this sucker...

This post will give readers a step-by-step guide to breaking into their SZ series (the cases on the SZ series are all the same), as well as how to replace to replace a drive in a Vista laptop without Vista screaming rape on the Internets back to the mothership. Last thing I need is freakin' Balmer showing up at my door with a baseball bat. (Oh, and you all know he'd do it.)

Ok, stuff you need:

Got it all, chief?

Oh sure, we could do this the safe way: copy the data on the old drive to a backup location, format the new drive and put a fresh copy of Vista on it, find all your applications from the old drive and re-install them on the new drive, etc.... Or, we can do this my way: just ghost the eff'ing thing to a new drive and hope for the best. Hey, that's what the Knob Creek is for...

So, fine - let's get this over with...

Step 1: Getting the new hard drive connected to the laptop

The new SATA drive has to be attached to the existing computer for ghosting - because the Seagate Momentus series SATA, and the SZ series doesn't have an external SATA port, you'll need a SATA external enclosure that translates SATA to USB-2 or Firewire. Rather than buy one specifically for the 2.5" drives, I took an enclosure that I had purchased last April for the Vista upgrade which houses 3.5" drives... since this wasn't going to be a permanent arrangement, who cares?

Step 2: Scaring up a Ghost...boo!

Ghosting is the act of making a bit-by-bit copy of one drive to another drive: the whole deal - boot sector, data, applications, file access tables...all of it. It's not a straight-forward operating system copy, and - in fact - can't be done from the native operating system at all. (Modern operation systems are always running, swapping cache data back and forth to the drive - meaning, that the OS itself is changing the contents of the drive as the drive is being copied.)

To successfully ghost, you need a application that will cause the system to boot up using a small little kernel that is resident in the computer memory - and doesn't fill up the drive with crap of its own. I picked Acronis Workstation 9.1, an application that was recommended to me last April for my RAID0 issue with my tower station, and did an amazing job. I upgraded the application to True Image Echo Workstation (or TI for short), and went to work.

Arconis starts in Vista, to give you a nice user interface, sets up a configuration file, and then reboots the computer - loading the smaller kernel in memory with the information from the configuration file.

When fired up, Acronis TI asks you what you want to do - although you may be tempted to hit "Backup," don't. What you really want to do is "Manage Hard Disks." Selecting this reveals a second screen that gives you the option of cloning your drive - cloning is another name for ghosting.

Very bad Acronis Action #1: Selecting on "Clone a Drive" reveals a very important screen that will make you swear like a sailor with the clap: "Automatic vs Manual" cloning. If you do what I did and pick "Automatic," Acronis TI will walk you through the appropriate steps for cloning, take just as long as an actual drive ghosting (about 3 hours) and....do absolutely nothing. Nice. Real eff'ing nice.

So...go with me here, and hit "Manual." Really. Trust me.

Hitting manual brings you through the drive selection process. You should see at least two drives listed here: the original drive still onboard the SZ370, and the new drive sitting on the enclosure that you have attached to the system. Selecting which one is the source and which one is the target is left as an exercise to the reader. As the new drive is larger that the original, Acronis TI's default action is to partition the new drive to its fullest capacity and then just move the contents of the old drive to comfortably into that space. You do, however, have the option at this point to play with partitions, making a second virtual drive.


After you've made your selections, Acronis TI examines both drives, reads the file allocation table on the old drive looking for sizing information, and sets up the boot partition on the old drive.

When it's done, it will present you with an information screen telling you what it plans to do, the number of steps it will take, and then asks you if your want to go ahead and do the deed.

At this point, take another shot of Knob Creek, and go for it. Your system will lock your old drive down so it can't be written to, and then commence a reboot.


Step 3: Creating the Clone...uh...Ghost...uh...Exact Same Drive

At this point, the SZ370 will reboot, but the operating system will not engage - instead the Acronis micro kernel will take over and start the ghosting process. Suck it up, grab a bigger glass, and pour a larger glass of the ol' Knob Creek, because you're here for a while. 3 hours to be exact. Sure - you could go play Halo3 on your XBox and not fret over the fact that the ghosting software that you purchased from a company you've never heard of before could be thrashing your hard drive. Well, I'm not that relaxed, ok? I'm just not...

The first think you'll see TI do is reanalysize all the partitions, defragment them, lock them down, and then check the new partitions on both drives to see if they are incapable of the clone process. Once this is done, the actual bit-by-bit cloning takes place. This process, referred to as "Operation 2 of 3" by Acronis TI is the step that takes the longest, and can be the most destructive to the drives.

Have a drink...be patient.




Very Bad Acronis Action #2: OK, here's another way that Acronis can waste three hours of your life through bad user experience testing. When the cloning is done, you get the "Happy, Happy, Joy, Joy!" screen - "Congratulations! You have successfully completed the hard drive cloning procedure."

Well, yes, that's very true. As is the "Press any key to reboot" suggestion at the bottom of the screen. What TI does not, in its exuberance at completing it's complicated ghosting task, tell you is this: If you reboot with the ghosted drive still attached, Vista will come back from the reboot, see two boot enabled drives, and remove the boot sector from the second drive - effectively effing up your ghosted edition. Three hours. Gone. Again.

Trust me here: when you get the congratulations screen immediately unplug the ghosted drive before touching any other bleeding button on your laptop. It's like religion at this point: just have faith that the ghosting worked without any proof whatsoever.

Step 3: Rip the Puppy Open!!

OK, all the namby-pamby software crap is out of the way, let's grab some manful, manly tools and get to work.

Like a helpless turtle found on the calm shores of life, flip the SZ over on its back. (Analogy too much?) Remove the battery before proceeding. You don't need to pull this off to work on the unit, but it makes it lighter, and removes the last remaining power source. (Uh, you did unplug the laptop, didn't you?)



There are four phillips-head screws that have to be removed at this point. These screws release the keyboard and the case covering on the front of the unit.








Flipping the unit back over, you can now ease the keyboard off. Between the F1 and F2 keys, and between the INSERT and DELETE keys, you will see barely visible little spring-loaded tabs that need to be eased back with the flathead screwdriver while you slowly pull the keyboard up. (This is where Niven and Pournelle's "Gripping Hand" would come in handy.)



At this point, the keyboard easily flips up. You can pull out the little flat ribbon cable connecting the keyboard to the motherboard, but its not necessary and not worth the risk of ripping the thin little cable. Simply lay the keyboard flat against the laptop's wrist-plate, or (preferably) up against the screen.

With the keyboard out of the way, three little screws that lock the wrist-plate in place are revealed. It's for these screws that you will need the little watch screwdriver - they are small, delicate and can be very easily stripped. Be careful taking them off, and - more importantly - be careful putting them back. Just tighten "to feel" when you do.

At this point, the wrist-guard can be slid away from the screen, revealing the hard drive. Like the keyboard, the wrist-guard has a few ribbon cables that connect the biometric scanner and the trackpad. It's not necessary to remove these either, as the wrist-plate can be flipped over and put on top of where the keyboard rested.



With the hard drive clearly revealed, yet another three screws (why always three, Sony??) need to removed in order to get the old drive out.







OK - scariest hardware move coming up: removing the ribbon cable to the drive. This thing is packed in there tight. Even with the drive screws removed, the drive cannot be lifted until the thin ribbon cable is removed. Unfortunately, Sony taped the damn ribbon cable from underneath. You need to spend a nice, relaxed, long period of time easing that cable off -- if you rip it, you're screwed. You'd have to order a new ribbon cable from Sony, and I have no dea what the part number is...so, uh, careful kids.

It's out! Excellent. All that's left to do now, is to take the two small rails off the side of the old drive, put them on the new one, and then your ready for Step 4!






Step 4: Uh...Repeat Step 3. Backwards.

Seriously. That's it. No pictures. No step by step. Just drink your Knob Creek, and put everything back together...I hope you were paying attention.

Step 5: Turn the Damn Thing On and Hope for the Best.

Ok - you've reassembled the unit, battery is back and the power is plugged in. If you've done everything right, you'll be greated by a final Acronis TI screen during the reboot cycle that says "Cloning Completed." Oh, happy day. The machine should now boot, with Vista being none the wiser.

After logging in, you will see Vista establish the drivers for the new hardware it has detected. At this point, one more reboot (the last one, I promise) needs to be performed, so that the drivers for the new drive can be firmly established. That's it, you're done.


Step 6: There is No Step 6

Seriously. You're done.

Let's prove it. A quick glance at the drive size reveals that you are now beefed up to a whopping 200gigs. (Well, minus 10Gigs for the boot sector and some lost clusters.)






What about the speed? Opening up the Vista performance rating test (Computer->Properties->Performance) will show the previous rating from the old drive - in my case, a paltry 4.6 out of 10.

Now, run the test again - it takes a few minutes. When it's done, you should see a noticeable increase in the disk transfer rate - in my case it went from 4.6 to 5.4...a 15% increase.

Done.






Aftermath

I did the upgrade about 2 weeks ago - so far, absolutely no problems, and nothing is lost. The unit generates a bit more heat (not much more, actually), but applications and data load noticeably faster - and the drive has plenty of space for all those Media Center videos I watch on the road.

OK, back to pontificating.

Thursday, May 1, 2008

Folding@Home: Johnny Molecular Biologist!

This afternoon I was having an IM conversation with someone about the Sony Playstation 3 - and after extolling a few of the system virtues, I off-handedly remarked that the real reason I bought it was that I like to cure cancer... which, of course, got the ??? in the IM window.

In a nutshell, here's the deal - if you're 900 years old like I am, you'll remember when SETI@Home was all the rage. (Ok, I'm being a dick. With 3Million users, SETI@Home is still all the rage.) In the mid-90's, a group of plucky astronomers at Berkeley realized that their distributed network funding required to complete the massively parallel channel search for signals from extraterrestrials wasn't going to come through, no matter how many Carl Sagan-inspired movies starring Jodie Foster appeared... however, all those new fangled 386's on those brand new cable modems were just sitting there downloading porn. Surely, there must be some better way of utilizing all of those spare cycles out in the world that didn't actually annoy the wife.

Installed as a screen saver on Windows boxes, SETI@Home was the first, true distributed computing application that made use of the real power of the internet: in very poetic turn, all of the people who were making use of the internet (which was created out of the ashes of DARPAnet, USEnet and EDUnet) to buy books from Amazon and pretend they are Brad Pitt in chat rooms, could now give back to the world by donating spare cycles on their computers in a massive cooperation effort to search for intelligent life beyond the earth.

Flash forward to 2000, and Prof. Vijay Pande at Stanford was having the same trouble: how to fund a computing effort that consumed massive quanitities of CPU cycles on a limited budget to crack the simulation of kinetics and thermodynamics of proteins and nucleic acids? The problem was this: Pande, and other molecular biologists, knew the sequences of atoms required to construct a protein if placed in the wrong order could contribute to amyloid-related illnesses such as Alzheimer's Disease. If they could change the protein back to the correct order, it would go far to helping cure the disease. But without a roadmap, molecular biologists have no idea how to change the sequencing to get a proper structure out of the protein. (It's a bit like knowing the pile of lego blocks scattered on the floor infront of you forms a race car, but the only way you have to put the car together is to try to assemble the blocks without a pattern.)

This act of bending and twisting and playing with the individual atoms of a protein molecule (a process called "folding") is a required "next step" to being able to predict the structure of the molecule. Fundamentally, understanding the process of protein folding — how biological molecules assemble themselves into a functional state — is one of the outstanding problems of molecular biology. Unfortunately, the myrad ways that a the atoms of a protein molecule can be arranged is legion.

Professor Pande, taking a cue from the extraterrestial hunters on the other side of San Francisco, created Folding@Home in 2000, which - also installed as a screen saver - used the untapped power of idle home PCs to brute force the folding problem: literally trying every possible combination of atomic sequencing in a protein molecule to see what works.
On September 16, 2007, the Folding@Home project officially attained a performance level higher than one petaFLOPS, becoming the first computing system of any kind to hit that kind of peak performance. (Uh, a petaFLOP is 1,000 TeraFlops...and, uh, a TeraFlop is 1,000 MegaFlops...which is...fast.)

However, it wasn't enough - there was still far more work to do...

...enter the First Person Shooter crowd. With rapid advances in GPU (Graphic Processing Unit) and CPU (Central Processing Unit) speeds, modern gaming consoles like Microsoft's XBox 360 and Sony's Playstation 3 have power to burn: the sustained speed of a PS3 at full tilt is 30,191MFlops. Pande and crew began to salivate. Most gamers only use their machines several hours a week, the rest of the time the machine remain idle or off.

Pande's group (Uh, really. His homies are now officially known as the Pande Group inside Stanford) approached Sony and struck a deal - the Sony Entertainment group would slap a user interface on the project worthy of the PS3 interface, the application couild be downloaded from the Sony Marketplace and installed by the user (sort of a forced "opt in"), and the user was free to run the app in the background when the system would otherwise be idle. There is a cost to the user for this, of course - the GPU and CPU would be always working, which chews up a bit more of the home electric bill. In the end, however, a typical user feels good about the project, Sony looks like an altrusitic uncle, and Pande gets to fold and unfold his proteins until the cows come home...which is good, because bovine spongiform encephalopathy (uh, mad cow disease) is one of the nuts that can be cracked by protein folding.

Below is a 5 minute video where I walk through the current version of Folding@Home on the Sony PS3.



Oh, by the way - the Folding@Home computing cluser, since going Ps3 in 2007 and hitting 1 million users in Febuary of 2008 - currently operates above 1 Petaflop at all times. That's a lot of Grand Theft Auto, kids.

As promised, here's the links of interest:

Vijay Pande's Folding@Home blog
Berkeley's SETI@Home blog
Wikipedia's explanation of the protein folding problem

Image at top of posting courtesy of
Peter G. Wolynes,
Department of Chemistry & Biochemistry, University of California - San Diego



Protein Folding Explained