last thursday, in the life of an Escaype forecaster
*LOUD ANNOYING MUSIC PLAYING*
After coding on the iOS app until 3AM Thursday night, I woke up to the sound of my alarm at 4:40AM. "Wait," I thought. "Why the heck did I set that stupid alarm?"
Oh, right. Sunrise.
I checked the satellite, saw nice high clouds but didn't look like 100/0, so I sent out a few forecast confirmations and went back to sleep for a couple hours. In my situation, I can only afford to mess up my sleep schedule if it's 90-100/0. That's my personal limit.
After less than four more hours of broken sleep, I woke up, quickly caught up on the groups after a sunrise that was a bit underwhelming in some places (though nice in SF). Seeing some pink skies in your photos put a smile on my face. It really did. I was sad to read that it didn't meet some people's hopes, but all I can do is apologize, and collect the data for our system, to look into what might have happened later. Turns out that we had worked with two models for this sunrise, one of which was updating the app, and another that was just on our back end. The one on our app called for a strong high cloud burn across most of northern California, and the other one called for mostly blue skies. What to trust? I'd been tracking the high clouds all night, and comparing their progress to what was expected in the models, and they looked on track to reach our areas of interest by the morning. In the last couple hours, the clouds reaching our latitude dissipated almost as quickly as they arrived. Seems like the actual answer was somewhere in between the two models.
It's not always that way, though.
Sunset started out looking like a pretty high-confidence blue sky forecast. This time, both models were forecasting 0% cloud cover over the whole greater Bay Area, and the forecasted atmospheric conditions looked highly unfavorable for cloud formation. NOAA's sky cover forecast was predicting at least 50% cloud cover at sunset in all of northern California, and around 30% cover all the way down to San Diego and inland in SoCal -- we aren't sure where those numbers came from, but we were very confident it would be cloudless from LA south at sunset. This forecast verified. (I was boarding a plane to LA, to visit home for my dad's birthday, so of course it would be blue bird skies!)
When my flight landed at 1pm, before even texting my mom to let her know I landed, I immediately re-loaded and scanned the satellite loops, and noticed that a massive patch of mid-level clouds, extending all the way from the bay area to the Oregon border, was present, and models had dissipated it all by noon. Even the models that initialized on that hour had failed to resolve it. But both models agreed that it would be clear by sunset.
"Weird," I thought. I've seen this happen before, though it hasn't been common that the models are THIS far off. Maybe they were just dissolving them a bit too early, I thought; let's give it a little time. The high clouds over the bay area, indeed, looked to be on track to move well out of the region by sunset.
But those mid-level clouds sure were strange. My mom took me to lunch in LA, but I was so distracted by the clouds 350 miles away that I could hardly listen or say anything. My phone stayed on the table and I was refreshing the satellites over and over. I felt really bad, but it's part of life. Reports and photos of beautiful clouds were streaming in through several groups, and you all were starting to ask about the sunset forecast. I've seen the models dissipate the clouds a bit too early before, so I didn't make any changes, as I did see signs that the big cloud mass was starting to break up.
Mom and I had plans to go hiking around LA this afternoon, but work came first. I told her I needed to stay home for a couple hours and get some things done. I felt really guilty, as I don't see my family often, but I had no choice. I needed to make sure I had everyone's back. I planned on coding the iOS app while checking on the satellites in the background, but conversations on Slack and careful tracking of the clouds took over, and I didn't get to the app. That's ok -- I would be able to do that tonight, after the clouds are gone for sure.
Well, 3PM came and went, and 3:45PM quickly approached. The clouds were beginning to break up, and fast. Huge blue holes were opening up on the satellite and webcams. It was impossible to predict where the next one would be, but it looked like only a couple patches would remain by sunset, if anything. I worried if I gave everyone the green light, most people would end up getting blue sky skunked, watching the clouds disappear before their eyes. But I knew I had to make my final call, or none of you would be able to make it anywhere for sunset.
Here's a summary of the information we had:
* all three weather models, all repeatedly forecasting unfavorable conditions for clouds at sunset (and favorable conditions earlier in the day, when there were indeed clouds)
* a NOAA forecast for ~50% sky cover in NorCal and ~30% in SoCal, though it was known to be blue bird in SoCal, and the clouds were quickly dissipating in NorCal and many areas were already down to 0% cover
* satellite imagery showing more clouds moving in, but huge blue holes opening up as they arrived and quickly broke up, seemingly at random
Given those points, what forecast would you have issued?
At that time, I issued an alert stating exactly what we knew: that the clouds would be patchy/localized, that they were in position to burn, and there was no threat of fog. I then updated the app data accordingly. I refuse to feed you all BS and fake things we don't know -- and that information is all we could reasonably say, given the conflicting data and high uncertainty in the outcome. This was no longer an organized cloud mass; it was quickly, randomly dissolving and leaving only small chunks behind. All the webcams now showed mostly clear skies, and although one last cloud mass was approaching SF from the north, it was already showing signs of breaking up.
By now, you all know what the final outcome was. Almost all of the clouds in the bay area did indeed disappear, with maximum 10 potential observed -- except for SF, where the one remaining cloud, directly overhead only for a brief period at sunset, blew up big. We would call that an 80-90/0 directly under the cloud mass, and 10/0 for those who had a view of it from afar.
Now, a little reflection.
Should the models have picked up on this remaining cloud mass? Absolutely. Yes. They were wrong about it. And they usually aren't.
Was our forecast wrong?
Depends on where. In 95% of the forecast area, it verified with blue skies (or quite close), and saved south bay folks the drive to Santa Cruz. The clouds didn't dissipate when they were expected to, but most were indeed gone by the required time.
However, the 5% of the area that the model didn't verify in, was in a very scenic, shootable area, and we know that was a possible opportunity for some of you. Tung and I don't just shake off, and we promise to use this information to make our service better.
Having said that, the NOAA sky cover forecast did not verify in 95% of the region, but it verified in the 5% around SF. (I'm not intending to bash anyone, BTW-- this is simply a comparison of different data feeds for the event, as this feed was mentioned in one of the groups today.)
After a night to think about things, and an hour-long-during-birthday-dinner phone conversation later, I still believe that, given the information I had, analyzing the most powerful weather models available to us, the info in my note was the best possible report I could have provided. But I realize it still may not have been enough to get anyone other than local SF folks out to shoot. And that leaves room for improvement.
What we're working on now behind-the-scenes is something really cool and a bit secret -- we anticipate revealing it in the next few weeks, provided it passes our preliminary tests. After that, we're working towards a system that better helps us compare current states to the model forecasts, so we can better track their progress and help provide you with even stronger guidance. This is not an easy task at all, and it has never been successfully, fully done before.
Anyway, this isn't meant to be a plea for forgiveness. And this kind of situation is not the norm. It's just a brief look into the kinds of decisions a weather forecaster is faced with on occasions when the models disagree or fail to verify. It's also a reminder that Mother Nature does throw curveballs. Lastly, it's a reminder that Tung and I are fully committed to helping you as much as we possibly can. And we always will be.
"The greatest glory in living lies not in never falling, but in rising every time we fall." - Ralph Waldo Emerson
(BTW, this is not a 'typical' day in the life of an Escaype forecaster. This is a more typical procedure.)