Archive for December, 2009

Adobe MAX 2009

Monday, December 21st, 2009

I have always been self-taught. My motto: “The answer is out there somewhere, all I have to do is find it”.

But, try as I might, this philosophy utterly failed when I tried to figure out how to use Adobe’s data services.

I realized this year that I needed to get out of my office and dive into the mainstream in order to avoid the long learning cycles that I was going through. What better place than Adobe MAX, where they offered many labs and lectures and thousands of attendees with whom I might mingle?

Unfortunately, I arrived at this decision about two weeks from the start of the convention. And I was in the beginning phases of a monster project  (see A Schlog through the trenches) and was trying to tie up loose ends of another major effort. And, I was having horrible headaches, presumably from the extra stress I was under. But, what the hell. I was tired of having excuses to delay my education. Damn the torpedoes and full speed ahead!

I had already missed the early registration window so I paid full price for admission and added an pre-conference lab to boot.  Yow!! Serious bucks there. I hope it’s worth it.

Now on to the plane/car. Flights were getting scarce, but I booked a commuter plane from PDX to LAX with a Hertz rent-a-car waiting for me when I arrived. I had hoped to use the refund from a ticket I’d cancelled earlier this year. And while they did honor the refund, it turned out to be about half the value of the original ticket after the “refund fee” was applied. Oh well… more dubloons down the ol’ tube.

Hotels in the area were even scarcer. I managed to book a room at the downtown $heraton. More on this later…

I then started filling out my schedule at the conference. Adobe created a neat web-app that facilitated registration to all of the labs and lectures they offered. Unfortunately, many of the ones I wanted were already full. I just signed up for the best mix I could find and moved on.

Since a few of the sessions were BYOL (bring your own laptop) labs, I realized I needed a new laptop. So, with only days left before the conference, I headed to Staples. There I found a pretty nice-lookin’ HP that had the required horsepower. No time to fuss, I just bought it and took it home to load it up with all of my development tools and current projects.

This was just a couple of weeks before the release of Windows 7, so I got Vista. This worried me, but I had no alternative considering the limited time frame. As it turned out, Vista was OK. It felt weird, but it didn’t crash every five minutes like I was led to believe.

I got everything ready the day I headed to the airport. The last-minute nature of this left me feeling like I’d forgotten something important. But, it was too late to worry about that as my wife was kissing me goodbye at the airport. Up , up and away!

The commuter plane was only 56 hours old, but things in the bathroom were already broken. Sheesh! But the flight was uneventful and we touched down on time in LA. The wait in line at Hertz seemed eternal as many customers felt obliged to tell their life story to the attendants. But I finally got my turn and just wanted the hell outta there. So, I asked no questions about the car they offered and just signed for the Mazda 5, which turned out to be a very nerdy family car that seated seven. I did not need this kind of car, but it actually was fun to drive and low-profile. Good enough.

I got to the hotel by merely pointing my car in the general direction of downtown and hoping for the best. Luck prevailed and I found the Sheraton with no circling or backtracking, and after paying the $500,000 per day parking fee (really $14 for guests) I flopped gratefully down in my room and took a nap.

After about an hour I got up and showered then headed down in search of Sushi. I found a nice spot right across the street named ‘Octopus’ where I got a couple of items and a beer. This helped my headache a little as I was beginning to relax. So, I headed back to my room to catch up on my work.

The labs and sessions had some source materials I needed to download, so I hooked into the room’s cable Internet connection (for an additional $14.95 per day !!##@!@#$!!) which was slow as molasses (apparently everyone else at the hotel was downloading porn movies).

At these speeds the many megabytes I needed to download would take all night! I gave up and decided to arrive early and install the software at the conference. I arrived a half hour ahead of time and found no one there to help me, so I waited til the start of the lab and there the facillitator gracefully installed all the needed stuff on my laptop and got me up to speed.

I was the only attendee who needed this kind of help as everyone else there already had their laptops running Tomcat and Flash Data Services and were familiar with developing in that environment. This wasn’t the last time I was to feel as if I were the dumbest guy there!

The lab was titled: Using Data Services to Power Flex Applications and was advertised as being for those with “a basic familiarity with Flex development”. But, I stumbled through the lab taking up most of the assistance of the speaker and his helpers along the way. I was overwhelmed by the speed of the presentation considering the amount of new stuff being covered. But by the end, I had a working app that basically was a standard CRUD personnel app using the TV show The Office characters as sample data. It automatically updated all clients when data was changed by any one client.

This is what I was hoping to learn at the conference. My Zilch game uses 1 second polling to keep all clients up to date. This design will bog down under high traffic (which luckily? I’ve never had on uZilch.com), so the techniques I learned could be directly applied to a real-world problem. I haven’t had time to do this yet, and the lessons I learned are fading away. But, the project is still there, on my laptop, for future reference should I ever get around to working on Zilch again.

This was just the pre-conference lab. The actual conference lay ahead, and I went back to my hotel room imagining what the next day would bring. Oddly, there was an adult entertainment convention being held at the same time at the LA Convention Center. I had imagined that I’d be greeted by a snickering, bespectacled, pocket-protected nerd with blushing tales of his foray to the ‘other side’. But, the boobs and dildos next door might just as well have been a thousand miles away. This conference was just too serious and demanding for anyone there to even notice that they had naughty neighbors just a building away.

But, it wasn’t all work. Adobe put together a couple of presentations at Nokia theater that were something like a cross between a rock concert and a mega-church service. I attended the general sessions which were emceed by Kevin Lynch, Adobe’s Chief Technology Officer. With “Silence is the Enemy” thundering on the enormous PA, the Nokia theater was transformed into the concert mainstay “bowl of stars” as Kevin hit the stage. Except here, instead of Bic lighters, the stars were made from countless iPhones and Blackberries being held high as they recorded the event.

After a welcome speech by Adobe CEO, Shantanu Narayen, and with no shortage of cheesy, but enjoyable, stage gimmicks, Lynch laid out Adobe’s latest achievements via the various program managers who gave mini-presentations on Flash Player 10.1, Flex 4, Cold Fusion 9, LCDS, Air 2.0, and a new tool called Flash Catalyst. We also watched presentations from large organizations such as Major League Baseball and FedEx that demonstrated their innovative uses for Adobe technology.

Adam Mollenkoph of FedEx demonstrated a real-time map-based tracking system for their fleet that made me wonder why the FAA wasn’t doing this kind of thing for the air traffic controllers. With maps that contained shaded areas denoting predicted arrival times for selected vehicles augmented by a wide variety of additional displays and controls, this app looked like a gamer’s dream.

The fact that the sports establishment now considers Flash Player to be an adequate medium for broadcasting games surely confirms everyone’s belief that “push” Internet has become a mainstream alternative to existing television technologies.

One presentation from the US Postal Service used what Lynch called “Augmented Reality” to draw a semi-transparent shipping box around an object. This way, the user can place things they might want to ship in view of the computer’s video camera and see how they would fit into the selection of shipping containers offered by USPS. And rotating in three axes too! Pertty cool stuff.

And to add even more cool to the Augmented Reality theme, Lynch introduced rock star, John Mayer, whose website features an app that uses the computer’s webcam to (presumably) inject the user into a Mayer music video, or rather, John Mayer into the user’s world. If you have a webcam and a special icon from his latest CD, check it out.  http://www.johnmayer.com/ar/

We were also treated to a preview of James Cameron’s new movie, Avatar; complete with 3-d glasses. I started out skeptical, but was soon eager to see the whole movie after being pulled into Avatar’s planet.

Here’s a photo of me and some other folks watching the preview…adobemaxavatar

I could go on about the keynote sessions, but you can actually watch them on Adobe TV. Here are the links:

http://tv.adobe.com/watch/max-2009-envision/max-2009-keynote-day-1/

http://tv.adobe.com/watch/max-2009-envision/max-2009-keynote-day-2/

As much fun as the keynote sessions were, I was here for the labs and lectures. During the pre-conference lab, the instructor mentioned Cairngorm several times. From Wikipedia: Cairngorm is one of the primary open source frameworks for application architecture in Adobe Flex. He mentioned that there was a lab later on that gave an overview of Cairngorm and three other frameworks. Oh goody! This is what I needed to learn. I figured my laptop was up to speed from the previous lab and things would go much more smoothly this time.

How wrong I was! Nothing worked when I got started at the lab. So, the same assistants came over and helped me again. They’d get me past a particular issue and I would try to get the rest of it up and running. Except, every step of the way, something was wrong or missing. Back came the assistant. He would semi-patiently move or copy this or that here or there or edit some configuration file somewhere else while the lecture proceeded.

Somewhere near the end of the Cairngorm lecture, I think I got it working and caught up to the beginning of the lecture. There was no hope of catching up with the speaker and the rest of the class who were following along on their properly-configured laptops and I hadn’t even started getting my computer ready for the next three lectures.

I was outta there. Before I suffered the assistants any more to help with my setup, I left the room feeling embarassed and defeated. Plopping down on my hotel room bed with my headache raging, I wondered if I should have stuck it out. But, I had work to do for my clients.

During the week I was in LA, I worked late almost every night on these projects. The CEO of one company was just now visiting the Brazil branch and wanted to see Portuguese translations of their website while he was down there. The environmental consultant wanted his file exchange site to generate a zip file at the server  for download from a Flex app. The animated greeting card investors were anxiously awaiting any news of progress on their complex Flex project.

Despite the crummy connection, I was able to make progress on all fronts. And subsequent lectures at the conference went much more smoothly. The presentations about the new features of Flex 4 (Flash Builder 4) were particularly interesting. The language made a sharp turn with this version and Flex programmers everywhere are going to have to make some significant changes to their coding styles.

Speaking of Flex programmers everywhere, Adobe MAX certainly delivered! One of my hopes was to meet some other Flex coders. But, when first faced with the lunch room tables, bustling with small groups intently talking about who knows what, I felt alone and intimidated. I was just some guy who showed up at this party and these were people who had been “in the fold” for years. They came in groups, sponsored by their employers. They were supposed to be there. Was I?

This sudden onset of shyness suprised me. But, the feeling was overwhelming. I slipped into an empty chair at a sparsely populated table and ate my lunch in silence hoping to hear a bit of conversation that I could relate to and perhaps would offer me a chance to chime in. Wow, Jr. High all over again.

Things were different at dinner, however, as I happeed upon the nice folks from Planet Bid, who welcomed me to their table and rescued me from my isolation. We wound up eating together for the rest of the conference and even though I don’t remember all the names, I do have fond memories of the smiling faces and lively talk that ensued.

After all, I’m very glad I attended Adobe MAX 2009. I plan to attend the 2010 conference, and I’ll be sure to register early and come prepared next time! See you there…

A schlog through the trenches

Thursday, December 10th, 2009

After several years of programming for the Internet, I have finally decided to start a blog. The theme ‘War Stories’ should be relevant to those who have spent time in the trenches.

This ‘war story’ is about a project which recently passed the Phase I milestone. Getting there was an epic battle.

It all started with the client asking me to re-develop the UI for an existing Flex app. It was an animated greeting card. I can’t go into details about the nature of the card, but it had a clever animation that could be controlled by the user. The client wanted to completely change the UI from their existing prototype, but underneath much of the workings of the app were to be the same. They also wanted to add a Text to Speech feature. The client also told me at the beginning that they wanted me because they were paying too much for the existing designers, who apparently were from a full-service agency. Ok, a FSA does charge more than me, and I’m not sure the client needed that kind of horsepower anyway. 

Well, I had done things in Flex that were somewhat similar to this, so I agreed to take a look. But, I needed the existing application source code in order to get a handle on the scope of the problem. When I received the source tree, I noticed two things right off.

1. The project had three components. A Flex project, a Flash project and a php project

2. The Flex project looked like it was built on top of some sort of MVC framework (model-view-controller) 

I started by looking into the server component. The php code referenced a database. Well, they probably had a document somewhere that described the database schema and what-not.

And, I have heard that it was easy to load Flash modules from Flex.

And, even though I have never developed a Flex project using an MVC framework, I have read that it is easier to re-implement an app’s UI in a project based on the model-view-controller structure.

And, there must be many sources for Text-to-Speech out there. What with all of the talking apps I have seen. (or have I really seen that many?) 

Anyway, I estimated what I thought was a reasonable bid and soon found myself deep in a war the likes of which I had never seen before. They often say: everything you know is wrong. Let’s see how that statement applied in my case… 

Let’s start with the server. Looking more closely at the php code, I found that there were no documents describing the project or the database. Just php code. Scanning that, I noticed references to many tables and other server resources. Oh great. That means I’d have to comb through who knows how much code to figure out how to set up a server so that I could replicate and work on the existing project. Rats! Maybe the previous developer will have those documents somewhere. 

Oh well… on to the Flex code. I figured that I’d run the app locally and use the previous developer’s server as referenced in the existing code. On first glance, it all looked good. Until I got to the animation, where I got a run-time error. It was complaining that it couldn’t coerce the loaded swf object to the expected interface. Double rats!

I looked into the Flash project and found no source code at all!

Oh well.. I’ll just fire off an email to the previous developer about the problems I’m having and I’m sure they’ll have a ready answer to all my questions. 

I suspect you are beginning to see what will happen soon… 

But, I needed to make some progress somewhere. So on to the UI. In the MVC architecture, the View component implements the look and feel of the website. I expected to be able to change this code to implement the desired features of the new UI. But, when I really looked into the requirements, I realized that the flow was different and the actual elements of the design had changed a little, plus new features were added like Text-to-Speech and a clear button. So, it became clear that, in order to use the existing code as a base for development, I’d also have to modify the Model and Controller portions of the code.

After some hours of walking through countless files, I realized that I had no idea of how their system worked. It was obviously based on a complex proprietary framework that had been adapted to this particular project. Double rats with nuts! 

Fast forward to after many emails to and from the previous developer (one where he suggested that I wasn’t skilled enough to work on this project). I finally got the Flash source code and not much else. So I compiled the swf using the code they sent and used that swf in the existing Flex project. Still the same problem. Aaarrrrggghhh!!!! 

OK, I’m in a foxhole and bombs are exploding all around. What should I do next? 

I decided to chuck the old project altogether and develop from scratch using the techniques I had used before. At least I’d be making visible progress!

So, I dove in and programmed like a madman for days and nights until I had a pretty good start on the new UI. I put it on my server (no database required, I just used xml) and the client was happy to see something after weeks of nothing but gloomy email. 

It looked like I had found a safe bunker as a base of operation. Scanning the battlefield, I knew my enemies were out there… ready for my eventual assault: TTS, a functional client-server interface, and most important, the animation. I knew the animation would prove to be a formidable foe, but I had no idea of the Goliath that was waiting. 

Despite the client’s obvious concern about the animation, I decided to tackle the Text-to-Speech problem. I found that there were fewer TTS service providers than you’d think. I found one at: http://www.neospeech.com/ that offered a web service where you posted the text and it would provide a url to the resulting speech file. Sounded cool! And they offered a SOAP interface. (always had wanted to try that). So, I signed up for the trial account and eventually cobbled together a Flex component that talked to the web service using the REST protocol. The Flex app just used the HTTPService component to post the request and to service the resulting reply. 

As it turned out, it wasn’t as simple as Request-with-text/Reply-with-speechfile-url. NeoSpeech actually replied with a status message. This message contained a result code and string (success, failure, etc…) and a conversion number (if success). If the result was success, you post again to another method using the conversion number in the request data. The reply to this post contained the actual url to the speech file.

So, after figuring all of this out, I was ready to play the file from Flex. Except, Flash player won’t play anything but mp3 files and NeoSpeech delivered results in .wav format. Hoping against hoppe to find a way for Flash to play a .wav file, I googled this subject until my eyes dropped out; but the answer was still the same.

Double rats with nuts and gerbils! 

A quick scan of the NeoSpeech documentation revealed that the trial version limited the format to .wav, but all these other neat formats would be added if a real account was purchased. So, I ordered a real account and would bill the client later. Now I could get the mp3 file I needed… drool drool. But, after carefully reading the list of available formats, I noticed that mp3 was NOT among them. I had to read the list several times. And after that, I even called NeoSpeech and begged them to add mp3 to the list. They said no and they weren’t planning this. (something about patent or copyright… whatever) 

I’m running out of rodents…. Now I saw the TTS monster for what it really was. And I felt weak and helpless. Surely I’d met my match. 

But, perhaps there was a way to convert a .wav file to a .mp3 file. Perhaps there was even a web service (kinda like NeoSpeech) that would do this for you!

I did not find a web service, but I found that many people were using a utility named FFMPEG to convert audio (and video) files. I searched for web hosting services that offered FFMPEG and found several. I settled on http://www.cirtexhosting.com. So, I opened a hosting account with Cirtex and got to the business of writing a php script that would do the conversion.

Oddly, an exhausting (if not exhaustive) search turned up no concrete examples of how to invoke ffmpeg from a php script. I posted a question to the help desk and I received the answer that they provided a path to ffmpeg and I was to use that to access the utility. They had no examples of how to do this, however. 

At some point along the way, I thought I had a glimmer of just what it meant to have the path to the utility. But I don’t really know now. Perhaps I (or someone else) will figure this out. In the meantime, the folks at Cirtex were nice enough to enable the system command in php for me. This is rare on a shared server, and I thank them for doing this! 

So, I got it working on a canned file, but now I needed to get it working from a url. It just so happens that php offers a function that works well for this. It is called stream_copy_to_stream. I created a new php file to service the Flex requests, dutifully compartmentalizing all of the details of the TTS server into the php interface file and away from the Flex app. And, it all worked! Most of the time anyway. It turned out that I had to keep requesting the status until I got a valid conversion. In the process of figuring this out, I abandoned the REST methodology and used their SOAP service. In order to do this I used php’s SoapClient (available only in certain versions of php).  

Goliath slain… On to the next adversary. 

The Flash-based animation was a sticky item to be sure. As loaded from the previous developer’s website, it wouldn’t link properly. The same was true when I loaded the swf I made from either my website or local directory. I was at a loss. I read up on loading Flash swfs from Flex, but nothing was helping. I decided to ask for help. I contacted a colleague I had worked with before (Tom Schreiber) and asked him if he (or anyone he knew) could help with my problem. He referred me to Russ Fergeson who suggested I move the swf file to my bin-debug/release folder. Voila! Thanks Russ! Now all I had to do was isolate the animation code from the old project and port it to my application. 

The MVC architecture is generally considered to be a “good thing” because it promotes a programming style that is modular and flexible. Unfortunately, if mis-applied, it turns an otherwise simple task into a programming nightmare. Instead of making a mini MVC module that was a component, the previous developers had intertwined the animation’s M, V and C elements into their overall MVC code. Now, I had to traverse countless other files and extract the relevant portions converting them to my simpler paradigm. This involved a great deal of experimentation. Kind of like hadron colliders, I was smashing code into code and watching the trails left in the aftermath of the destruction.

And, just to make things interesting, my computer got the swine flu and puked all over my office. Luckily I was able to transfer my projects to the laptop and continue on while the main computer went to the hospital. 

It came down to just never giving up. Eventually, I had a component that I could use from my application. Things went pretty smoothly as I added controls of the animation almost verbatim from the old code. A sure sign things were going well. I was about ready to relax a little when the next bomb hit. 

I had been having a problem with my server-side code where it would mysteriously become unavailable for about 20 minutes. This sounded like a session issue, so I re-wrote my code to eliminate use of session. I thought this had fixed the thing, but all of a sudden I couldn’t access the server at all. Not even via ftp! At this point, I had the application residing on one server (mine) and the TTS proxy on another (Cirtex). It seemed reasonable to port all my code to the Cirtex server and try it there. When I did, it seemed to work once. Then that server became unavailable! 

What had I done? I scoured my server-side code for that fatal flaw. Nothing I knew suggested that this code would cause a problem. Had I been writing deadly code all along? Somehow missing the crucial ingredient that all “real” php programmers knew to use?  

Of course I had just sent an email to the client announcing an imminent demo. Oh, the irony! This was as classic as the Illiad. Now I was lying wounded in the mud, with tanks rolling in, and I had nothing. But, hope springs eternal! 

Ignoring the futility of trying the same thing again a third time, I decided to open yet another hosting account. At least I would be able to do basic experiments in php server technology, if for no other reason than remedial punishment. 

But as I was running the app again using the new server, I noticed a funny thing. A new icon in my system tray was blinking red bars at me. It was the recently installed Firewall: Comodo. (I had just gotten my computer back from the shop where they installed it as part of their service).

So I clicked it. 

To my surprise, it showed that my computer had spawned hundreds of IP sessions and was in the process of spawning hundreds more. Yikes!! At first I thought that I was still infected, but after looking at the destination IP, I realized that it was pointing to the server I was using for the project. Oooohhh… 

I closed my app and they stopped spawning. 

OK, so it wasn’t my server-side code after all. I was fighting on the wrong side. The real enemy lay in the Flex code. So what had I done to cause such a problem? 

Well, when my server problems started, I had written a component based on HTTPService whose send method was re-invoked by an occurrence of the fault method… if the count was less than some constant. (I used 5). Aha! That must be the problem somehow. So I ripped this code out. 

But the problem remained. Now what? When I get flummoxed (which is quite often really) I use a great comparison utility called BeyondCompare. I save my work folder to a backup drive and add an increasing number to the backup’s file name. Yeah, I know this is kindergarten level version control. I have a CVSdude account. But this method works well and is much faster than using cvs for my purposes. I do use CVSdude for when I am on a team however. 

But, anyway… walking back in time via the BeyondCompare file comparisons, I noticed a small, seemingly innocuous change in a file that I had overlooked before. I had a problem where a Flex component was loading its data file on the creationComplete event (which only happens once at component creation time), but sometimes the data would change and when the user returned focus to the component, it didn’t show the changed data. So, I had reasoned, add a request for the data when the component gains focus. 

This was the culprit. I really don’t know how, but doing this caused the Flex app to keep spawning request after request, with the only symptom being that the server would get slower and more grouchy until it would sign off altogether. 

I came up with a more sane way of fixing the data problem which (thankfully?) I won’t go into here. But, now, my original server is down and my TTS server is down and my new server is bombing on file upload (presumably a configuration issue, but who needs this?). And the client is sending emails asking about the demo. 

So I get onto live chat with my server company jodohost.com. I ask them to check my account and they say everything’s fine. They can access the website from their browsers. I thought it might be my ISP. So I tried my Blackberry, and sure enough, I connected. 

So I call up my ISP gorge.net. I figure that they had some automatic system that detected crazy clients who spawn thousands of connections. But, they said no, they didn’t have any blocks on my account. 

Well, it had to be somewhere!  

I went back to Jodohost. This time I asked if there was a block on my specific IP. And sure enough, they had sensed the aberrant number of requests from my computer and had blocked that IP. 

After apologizing and promising to do my best to avoid making programming errors, they re-enabled my IP and I was back in business. And, just around that time, Cirtex got their server back online. I am still unsure if I had anything to do with their server crash. 

The demo went out about a week after Thanksgiving, the target date. But, it looked pretty good and the client seemed happy. If only they knew what I went through to get it to them! 

Shannon Vance

12/11/2009