Things I learned as a web developer from the Comic-con ticketing fiasco

Today was the big day – at 9AM pacific, tickets went on sale for San Diego Comic-con 2013.  Unfortunately, their web developers had a few bugs in the software, one of which bit me bad enough it killed my ability to secure tickets.

As a web developer it hit me about 15 minutes in as to why I wasn’t moving up in line (and the “customer service” line was busy the entire time). Once I opened the Developer Console in Google Chrome and saw the JavaScript errors I knew it was over.

  1. Store all static assets on a CDN. This is the bug that bit me – there were some JavaScript resources that it was trying to pull from the server that timed out with 503 errors (service unavailable, usually due to demand). If these files were stored on a CDN all over North America & the world, it wouldn’t have been an issue. Even if they were hosted within the same “cloud” instance, if one critical piece of the cloud goes down, it would take all those requests with it. Why is it a big deal if the JavaScript doesn’t load? Because the JavaScript had the code that would refresh your position in line, and when it was your turn, redirect you to the portal to buy the badges. So what ended up happening to me was that I was #22,397 in line, but I never advanced in line because the JS that was responsible for moving me up in line never loaded and never executed.
  2. If the next page is going to take a while to load, tell people. The other big issue I saw the white page of mystery. Basically, after you clicked the big green “GO” button at 9AM, if you made it into the EPIC (the name of their ticketing system) waiting room, the screen would be blank for a while, for me it was about 90-120 seconds. But then the screen would come up and tell you what place you were in line, what not to do (don’t refresh the page, etc). But people didn’t know what to do with this infinitely loading page. What I would have done was serve a static HTML page, and then used AJAX to retrieve that person’s place in line (via the session key), fill in all the details and then let them wait it out until they’re at the front of the line.
  3. Sometimes, just putting it in on a cloud-based system isn’t enough. Sometimes you might need more than one cloud-based backend. Partition the servers (e.g. 20,000 tickets on each) and go from there. Likewise, multiple card payment services might need to be used in order to process card swipes quickly (you might need 3 or 4 different providers, and you just switch off between them based on a hash of an order ID #). 

So thats what I learned as a web developer to cope with high demand events.

Now as a nerd, I’m still pretty pissed off I wasn’t able to get a ticket, even though I was lucky enough to make it into the waiting room. Some time around May they’ll release some more badges, but likely the scramble for them will be more desperate and furious than this one.

UNLV Now’s Wishful Thinking

UNLV Now – the master plan that is designed to bring an indoor covered football stadium to the UNLV campus – is slowly making progress towards reality. The problem is that their envisioned reality doesn’t seem to match up with the real world (Executive Summary, full report – both in PDF).

The most problematic thing is they’re selling the stadium under the guise of having 15 major events per year to bring in additional tax revenue. There list is as follows, with critical commentary about each idea.

1. PAC-12 Football Conference Championship Game (December)

They start the list off well – this is something that could easily happen. Along with one neutral-site PAC-12 game each year (earlier in the season). (+1)

2. NFL Exhibition game (August)

The report authors are on crack. The NFL hates Las Vegas and legalized sports betting with a passion, and there is no way the league would allow a game to be held one mile from The Strip.

3. New College Football Bowl Game (December)

While I think we could get a better bowl game, its not going to be a big blockbuster game and definitely wont be a BCS game (the NCAA doesn’t like Vegas/betting much either). (+1)

4. Neutral Site College Football Game (Fall)

As mentioned above in #1, this is likely to happen. (+1)

5. Second Neutral Site College Football Game (Fall)

I’m not so sure about a second game – we need to get one game first, establish it, and make sure that the teams coming to Vegas are happy with the results before we start thinking we can get a second. We’ve been talking about a second NASCAR race here for a long time, but that hasn’t happened yet (though we have gotten the end of the year celebration). That said it could happen – Las Vegas has been very successful in recruiting west coast basketball conferences to host their games here – neutral court as well as being an attractive tourist destination with a variety of hotels and prices available to different kinds of fans. (+0.5)

6. International Soccer Festival (Summer)

This would be good, we’ve had a few soccer games here before, and I’m sure we can attract a lot of foreign visitors (thanks McCarran for the new T3, its very nice!). (+1)

7. Electronic Music Festival (2 to 3 days, Summer)

While it would definitely be easier for the EDM kids to go by shuttle bus to the new stadium instead of the speedway, there are several reasons I don’t think this will work.

1) Most large music festivals are held outdoors and in large flat areas – not in an indoor stadium. EDC, Coachella, Bonaroo, Glastonbury – all outdoor festivals. This project doesn’t have large grassy areas for festivals to be held.

2) Capacity – I don’t think you’ll be able to accommodate 100,000 people per night in a 55,000 seat arena, even if you open up most of the field area. Plus, EDC has multiple stages, not possible in a football stadium – you’d have to setup other stages outside the stadium in the adjacent parking lots and even then I don’t think they’re big enough for EDC-level of attendees.

The LVMS is the only places in town, other than the Convention Center, that could host the EDC. If ticket prices and how quick they sell out are any indication, EDC would like to grow, not shrink, their event here in Vegas.

8. Country Music Festival (either with ACM weekend or another time) 2-3 days

Yup, this would be a fan favorite! (+1)

9. UFC International Fight Week (Summer)

Another good event – UFC could fill most of the 55,000 seats assuming the stadium has a Cowboys Stadium-like video system. (+1)

10. Tour Concert (Summer)
11. Tour Concert (second, Summer)

Two major tour stops are a reasonable assumption, the question is what bands or performers can you think that would sell more than 25K tickets to a stadium show in Vegas? U2. Whatever the latest Disney-manufactured sensation is. Thats about it. Plus, the casinos hold the monopoly on most concerts nowadays (either MGM Grand or Mandalay Bay for very large events). (+2)

12. Winter Kick Soccer Festival (February)

Two soccer festivals in one year? Lets not push our luck.

13. Rock Music Festival (Summer)

This faces the same problems as the EDC above – the fact that it is indoors and doesn’t have a lot of room around the stadium for multiple stages and multiple bands going at once. You could have one stage at the football stadium and one at the T&M, but that goes back to them both being indoor setups, not traditional music festival setups.

14. X Games (Summer)

Possible, but I don’t see Vegas getting it every year – maybe every other year or third year in rotation with LA and San Diego. (+0.5)

15. NFL Pro Bowl (January)

See #2 above – the NFL hates our guts. Not happening.

16. MWC Football Championship Game (December)

The football side of the MWC is a joke. Sorry, but considering how all the quality teams are leaving to go to other conferences, it just shows we’re the ugly kid at the prom. This will happen in future years because the MWC has 10 football teams, but that doesn’t mean its going to attract a lot of ticket sales and major national attention (e.g. TV ratings). (+0.5)

17. NFR Closing Event (December)

Yup! I would just hold the entire NFR at the stadium, one of the halves of the bowl and then sell 25K tickets, much like they do with playing basketball games in football stadiums. It might not be the best fit but its more seats than the T&M allows for now. (+2.5)

Also, they listed a possibility of other events coming to Vegas based on talks with the LVCVA.

18. Wrestlemania

Do people still watch wrestling? Do they know its fake? Its like a soap opera for angsty kids and teenagers who cant control their anger so they have to live out violence vicariously. I guess we could get one occasionally, but wrestling doesn’t seem that big here in Vegas – why not just go all-in and watch UFC? (+0.1)

19. Republican or Democratic National Convention

I highly doubt either party wants to hold their convention in the “city of sin”, where “what happens here, stays here”. Too much fodder for their opponent.

20. NCAA Final Four Basketball Championships

No. Again, the NCAA hates us. Not happening at all. Never.

21. Comic-Con

I am a nerd and would love Comicon to be in Vegas, however 1) I don’t see how a stadium helps when we already have the logistical critical mass necessary in terms of hotel rooms and convention space and 2) its gone from nerdy things to pop-culture things in the last few years and I don’t see how Vegas fits into that (its gone Hollywood, and its a 2 hour drive to SD, and a 4-5 hour drive to Vegas). That said, they could finally hold a Firefly/Serenity panel at the stadium and not reach maximum occupancy! 😉

22. Boxing

Yes, but no one really cares about boxing unless there is a fight between Mayweather and Pacquiáo. And who knows when that’ll actually happen. (+0.1)

23. MLS All-Star Game

Yes, but again I don’t see Vegas getting one of these every year – just a once every 5-10 years thing. (+0.1)

24. Corporate Events

Sure, but I don’t think we’ll see a lot of them filling up the joint, especially in light of the ridiculous amount of convention space already present in Las Vegas. (+0.1)

If you count up the list of all the annual events, and partial (every other year or every third year) events, you get a total of 12.5 events annually, below the 15 they are targeting for their tax revenue generation promises. So take almost 20% off their tax revenue estimates before you take those numbers and plug them in elsewhere.

Las Vegas needs a nice, “major league” stadium. But don’t use inflated numbers to justify it, it’ll only come back and bite you in the ass. Besides, after looking at the UNLV master plan, there is no way they would have adequate parking for the stadium’s events – the master plan shows taking away more than half of the T&M parking and then adding the stadium to that, with only a few thousand additional parking spots added back into the mix. So the new stadium would have less parking that the current T&M has now. Good luck with that!

I-11: Building what we can now

In response to an article about getting the new I-11 highway done quicker, there are things we can do right now to get the project moving along. First is the Boulder City Bypass. At around $400M total (phase 1 and 2) it is a substantial amount of work, most of which is cutting through the El Dorado mountains east of BC. The good news is that the environmental work is already complete and the first phase of the project is about to get under way. Short of any lawsuits from environmental groups, work on the final design and construction could start pronto.

What we need is about $360M over the course of 3-4 years for a design-build phase 2. But this is a good starting point. Think of this project as the cornerstone to the entire I-11 corridor. While work is done on this project, the next project in the line, the Kingman I-11 & I-40 interchange is already under study and plan to finish their EIS in summer of 2013 be ready for detailed design and construction (pending funding). Smaller work, like replacing at-grade crossings with interchanges and frontage roads, should be packaged along with adjacent major work to reduce the number of times projects have to go out to bid and to consolidate design work. Design-build should be used throughout the project to speed things up (the only thing I don’t like about design build is that things get cut – a ramp from the D-street interchange on I-15 to 95 west was originally in the plans but was removed, and also a ramp from the frontage roads on I-15 to Blue Diamond east was planned but removed in design — both of which were useful to me in my travels).

The entire project, from Las Vegas to Phoenix can take 15 years instead of 30+ if we pipeline these projects together. When one project is in the design phase the next two are in EIS. We don’t need (and certainly won’t get given the current political climate) all 15 projects to start up at once to complete the entire shot in 5-7 years, nor do we need an abridgment of the EPA regulations, what we need is commitment from government and a funding schedule that is reliable. We need money to accomplish this (though I’m no help – I’ve paid $0 in gas taxes since I bought my Chevy Volt over three months ago, currently around 500MPG). We need $250M per year for the next 15 years to get I-11 substantially completed. We need the current Congress to approve this funding roadmap and commit to it, and for future Congresses to keep their damn dirty hands off it.

Good luck with that.

HARP 2.0 – Part 3: Hitting a Roadblock

The short version of this story is that I’ve hit a roadblock in refinancing my underwater house under HARP 2.0. I cant get approval from Fannie Mae, despite stellar credit ratings and a strong balance sheet. Fannie Mae has five levels of HARP 2.0 ratings – Approved, Extended Approval I, II, and III, and Ineligible. Most places are currently only refinancing Approved and EA-I ratings. Many folks are getting EA-III approvals however, and there is really no recourse – Fannie Mae wont be able to tell you why specifically you’re getting that rating since its a software program that is spitting out only an answer with no justification as to why.

So now I sit and wait. I have two options – wait for mortgage companies to start accepting ratings below EA-I, or wait a few months and try again and see if my rating has changed between now and then.

HARP 2.0 – Part 2: Finding a lender

So you’ve read part 1 and figured out that you’re qualified for the HARP 2.0 program. Good, now comes the more difficult part of the process, finding a lender that will refinance you. This can be easy or difficult depending on how underwater you are on your house, and whether or not your current mortgage servicer offers refinancing programs. I was in the difficult position of having two strikes against me – my current mortgage servicer does not lend (they’re strictly a mortgage servicer) so I had to find a different lender and my Loan-to-Value (LTV) ratio was above 125%. The only thing that could have made that worse would be if I paid PMI when I bought the house or if I had a second mortgage (neither of which were a problem for me).

Your Existing Lender

If your current mortgage servicer offers programs to refinance your loan, this process will go much smoother. You aren’t required to refinance with them, but it is a good place to start looking and comparing rates. If your LTV is below 125% then it’ll be easy to shop around, but if its above 125% you might be stuck with your current servicer with only a few other options to check out.

A Different Lender

If your current mortgage doesn’t offer programs to refinance (like mine) you’re stuck having to find a new company to refinance with. This becomes especially difficult if your LTV ratio is above 125% as many lenders aren’t refinancing Fannie Mae and Freddie Mac backed loans above 125% even though March 15th has passed and the new software (Desktop Underwriter or DU Refi Plus) has been rolled out to everyone.

Also in my case and the case for those refinancing at above 125% LTV, only one loan program through Fannie Mae is currently being offered – 30 year fixed rate. No other programs (20, 15 year) are currently being offered for those above 125% LTV. They may be rolled out in the next few months (June 1st was a date I had heard but I don’t know if thats accurate).

In my experience, I had tried the major banks and none of them were accepting customers from other loan servicers with LTV ratios above 125%. Even major online mortgage companies were capped at 125% for the time being. I had to contact about 12 banks before I was able to find two that would refinance me in my current situation. (names intentionally omitted until I’m done with the process)

Getting Multiple Quotes

Finding multiple lenders is important, as you can play them off each other to get a better rate. When I had first got my house, I had two rate quotes, and one lender was able to match the other’s lower rate and pay for part of the closing costs, instead of just having the lowest rate.

Moving Forward

The last step is getting all the documents from both the lender and that you’ll need to complete the underwriting process.

Thats it for part two. As I move through the process I’ll post the third (and presumably final) part when my loan closes and I get a final figure for how much my payment will go down a month, and what I have to do to amortize at the same rate as my current loan.

If I had $500 to spend on content per year…

I thought about this the other day, about how most websites have garbage ads on the sides, most of which are complete fucking scams. One of the websites I read I subscribe for $50/yr. So if I had 10 websites, $500 a year, to spend on content subscriptions where would I spend it?

  1. Ars Technica (which I already subscribe to for $50/yr)
  2. Las Vegas Sun
  3. Green Car Congress
  4. AnandTech
  5. The Verge
  6. BoingBoing
  7. Five Thirty Eight (a blog on the NY Times website, I’d subscribe to the entire site just for this and the Paul Krugman columns)
  8. Fareed Zakaria GPS (a blog on CNN’s website)
  9. Fierce Wireless/Fierce Broadband Wireless
  10. Reddit (not a news site, but still I’d pay)

Way OT: Scott Pilgrim Movie Timeline

This is really really really off topic. But I was really bored tonight and without a car.
So I determined the movie version of Scott Pilgrim takes place over at least three weeks (from late March until the third week of April). There are two distinct Fridays mentioned in the movie – Julie’s party and the concert where Sex Bob-omb opens for The Clash at Demonhead. Then after that, the next firm day of the week found in the movie is Thursday (from the AMP v AMP poster against the Katayanagi twins), so thats at least 20 days. Plus the days before Julie’s party. The shortest time span is March 30-April 21, with slightly longer timespans possible.


Earlier in the week:
Hanging out at Stephen Still’s house, band practice

Next day:
Hang out with Knives after her school day is over – arcade, thrift shop, music store, Scott’s house (outside)
Ramona dream

Next day: (this might be Friday, just earlier in the day)
See Ramona
Band practice

Friday April 1, 2005 *
Julie’s party
Scott talks to Ramona

Email from Patel
Scott orders amazon movies
Hanging out with Knives again

Saturday or Sunday afternoon:
Band practice

Amazon delivery
Date with Ramona

Wednesday morning:
Wake up with Ramona, invite her to Rockit “tonight”

Rockit concert
Fight with Patel
Spent night with Ramona (“first and a half base”)

Thursday April 7 *
Break up with Knives
Band practice
Dinner with Ramona
Fight with Lucas Lee

(nearly a week goes by)

Envy calls
Knives shows up
Encounter with Roxy
Coffee shop – encounter with Envy, Ramona

Thursday April 14th *
Wake up with Walace, Other Scott, Jimmy
Band practice

Open for Clash At Demonhead
Fight with Todd Ingram

Late Friday night
Pizza pizza
After party
Fight with Roxy
Fight with Ramona

Tuesday [note: not shown in movie, mentioned by Stephen on Thursday, April 14]
2nd round of the TIBB, presumably Sex Bob-omb wins

Sometime between Saturday and Thursday
Band practice

Thursday April 21st **
Concert/battle Katayanagi twins

Later that night (????)
Gideon battle

* Dates extrapolated from the only hard date in the movie, see **

** This is the only hard date in the movie – on the AMP v AMP poster, the date says Thursday, April 21st, 10:00PM Sharp – FWIW, the two closest years with a Thursday April 21st are 2005 and 2011. Every other date is calculated from this poster


I have too much fucking time on my hands.

[Update 1: added 2nd round of TIBB]

Past Predictions (from 5 years ago)…

With the news that Plastic is shutting down next month, I went through my list of comments and found this gem. Not bad.

2006 Technology Predictions:

1. Mac computers will see a surge in sales as models switch to Intel processors. This is in part due to the ability to run Windows applications easily from Mac OS with only a little performance hit. Customers who ordered their new Macs will be disappointed by the wait times for their new hardware to arrive, but The Steve’s RDF will make it all better. Apple’s shares split as they hit 100 towards the end of the year. [This just about happened! AAPL was between $85-90, though the shares still haven’t split]

2. Web 2.0 hype will fade towards the end of the year, however the underlying technology will still hang around and provide neat new innovations. [Web 2.0 hype ended as soon as Apple introduced the iPhone in January 2007, but AJAX and all the useful Javascript frameworks we take for granted today came out 2006]

3. TiVo will have a comeback as they fight for fair-use (not having to buy the same content in DVD, UMD and iPod video just to have it in all those places) and unveil a new Hi-Def TiVo with new integrated features like Podcasting, IPTV (certain codecs — perhaps MPEG 4 AVC/H264), chat and games (better integration than we see in HME now). [TiVo never fought much for fair-use, but we did get an HD TiVo in September 2006 – and features like Podcasting did come to the HD and SD units)

4. HDTV penetration continues — Cheap CRT RPTVs dominate the low end of the market, while DLPs take over the mainstream and plasmas and LCDs fight over the high end 50″+ thin-and-flat segment. 1080P becomes standard for TVs> 50″ and 720P below 50″, even through there is no source material at 1080P. [This sort of happened, DLPs had their time in 2005-2007 before flat screens really came to dominate the market]

5. MS Vista debuts at the end of the year to mixed reviews. We waited 5 years for this? With all the main “innovative” features ripped out, all we end up with is XP Release 2 (from the users perspective). However the architecture underneath is more solid and stable, and the successor to Vista is announced and might actually contian most, if not all the missing features. [HAH! I was soooo right about Vista being a disappointment, and Windows 7 actually being the refined, complete Vista]

6. Google continues its dominance. Products like GMail start to exit beta and feature more ads. Some products like Orkut are put out to pasture. [Orkut might still be alive but no one in the US uses it, it got taken over by rowdy Brazilians]

7. PS3 Delayed to summer, however Nintendo’s Revolution is the hit of the fall, at only $250 and the ability to play the entire back catalog of first-party Nintendo games (Mario, Zelda, Metroid, etc) for the NES, SNES, N64 and GC available on-demand via a download for all but GC (GC discs will play in the Revolution). For $25/mo you can play unlimited games. I’m not sure about this part: first-party Sega games along with some third party games for the Master System, Genesis and 32X are available as well. (woooo Phantasy Star!) [The PS3 was delayed until the fall, longer than I had expected, and the Nintendo Wii has sold the most units out of it, the 360, and PS3; also I have played Phantasy Star on the Wii and lost a lot of time to it]

Future Visions Pt.1

As CES goes on in town this weekend, seeing all the new products and technologies introduced gives me a few ideas on what might be possible in 10 or 15 years. I’ll cover a few of those ideas here over the next few days.

Idea 1: The idea of a personal car is outmoded

One of the bigger announcements at CES is Ford’s all-electric Focus. While this is good for now, the future of transportation isn’t limited to just personal vehicles.

Imagine Sunday night in the year 2030. You’ve got plans laid out for work this week. You get up each day at to be at work by 7, and you leave work at 5 at night. You share your weeks calendar with the AUTOMAT car scheduling system allowing them to know what time you have to be at work, any errands you have to run, and even suggesting you stop by the flower shop for your wife since your anniversary is Thursday.

The massive computers that run the system tell you what time a car will be by your house to pick you up in the morning each day. Yes, the car will pick you up. Using an advanced version of Google’s self-driving car technology, the cars will be able to autonomously drive you around the city, including from their nightly storage and recharge areas (possibly mall or supermarket parking lots) to your home. Think of it like car sharing on steriods. You can even get a discount for carpooling with friends and co-workers.

So what could cars look like if they’re going to be almost crash-proof and typically carrying only one occupant? Far lighter and cheaper than they can be made today. This, compounded with advanced battery technologies, will allow cars to drive hundreds of miles all day at higher speeds without having to pause for a recharge.

You could even sign your children up at age 8. No more needing to take your kids to soccer practice, since they can take themselves in one of these vehicles. They’ll have their own smartphone to control when they need to be picked up if practice ends early.

As cars become more connected, monitoring the cars performance and their occupants will become trivial. Computer systems could easily pinpoint malfunctioning cars before they break down and direct them to service or have replacement cars to your location quickly. The system could even notify you if you accidentally leave your gym bag or cell phone in the car. The vehicle would also have a self-defense system – it could determine if a human driven car was at fault in an accident, or capture the license plate of someone who may have hit the car in the parking lot.

Out of town trips? No big deal. Larger vehicles with cargo space for luggage are also available. Higher speeds would also be available on the highway to get you to your destination faster. Long day trips or multiple day trips might require users to swap cars (or stop to swap batteries) if they want to drive uninterrupted to their destination without stopping overnight at a hotel to sleep.

Larger societal implications

  • Reductions in number of auto fatalities. There still might be many caused by outside circumstances – pedestrians and cyclists jumping out in front of cars without looking, human driven cars, etc.
  • Taxi drivers and chauffeurs are put out of work – no one needs a human to drive a car anymore
  • Auto body and car repair shops see massive reductions in the amount of work they have due to fewer moving parts and more proactive maintenance schedules
  • Increased mobility for kids – ages 8 to 16 can now get around without relying on their parents (though safeguards in the system will be set to require parent permission to reserve a car)
  • Reduced number of cars produced every year because of vehicle sharing – how does this impact manufacturing?
  • Less differentiation between car manufacturers – people still might want to spend the money to reserve a Lexus or BMW but most of the safety concerns of larger and more expensive vehicles will have gone by the wayside
  • Increased patronage of bars – if you don’t worry about driving drunk, you can drink more ;)

So thats really just the first idea I’ve had taking up room in my head as I’ve read all the news pouring out of CES. Hopefully I can put together a few more articles the next few days.