NJ Strong Weather Forum
Would you like to react to this message? Create an account in a few clicks or log in to continue.

Roidzilla 2021 Model Performance

Go down

Roidzilla 2021 Model Performance Empty Roidzilla 2021 Model Performance

Post by Frank_Wx Sun Feb 07, 2021 2:45 pm

I thought this would be a fun exercise and spark some debate. I tried to pull every model's 12z run from Sunday, January 31st and their forecasted snow maps through Tuesday, February 2nd. I realize some areas were already seeing overrunning snow 12z Sunday, but I figured a 12z snapshot on January 31st would still capture majority of the overrunning plus obviously the entire duration of the main event that took place late Sunday all the way into Tuesday.

First up - RGEM

Roidzilla 2021 Model Performance RGEM-Verification

I felt the RGEM was too aggressive. DC to SNJ to CNJ did not get nearly as much as it predicted, and it did not do well in west-central NJ and east-central PA. Its jackpot zone was too expansive, but it got the 'general' jackpot area correct across NW NJ/NEPA. A little underdone in NE NJ. I expected better from a short range model. Grade: C

Second up - 12KM NAM

Roidzilla 2021 Model Performance NAM-Verification

Impressive. It predicted the lower amounts in the southern portions of the region and was also good in outlining the jackpot area. It did under-call central Long Island, but I am not sure any model predicted the subsidence over that part of the island. Grade: A. Bravo, bravo!

Third up - 3KM NAM

Roidzilla 2021 Model Performance NAM-3-KM-Verification

Ehhh it almost looks like it forgot east and west. For real, the small jackpot area it showed in Monmouth County is nearly the same latitude as Allentown, PA which in reality saw close to 2 feet while Monmouth was mostly below 2 feet. NW NJ did better than NE NJ. I don't know, this is some weird ****. Grade: C

Fourth up - HRDPS

Roidzilla 2021 Model Performance HRDPS-Verification

Ma che cazz? I know this is 'Crapola' but I checked and the 10:1 map is very similar to this one. This is the saddest performance I've ever seen. Grade: F

Fifth up - CMC

Roidzilla 2021 Model Performance CMC-Verification

Also on the aggressive side. Especially toward SE NY and SW CT. Also it seemed to under-call north-central NJ area. I don't think it was as bad as the RGEM, though. Grade: B

Sixth up - Icon

Roidzilla 2021 Model Performance Icon-Verification

Similar to the 3KM NAM, seems this model could not pinpoint the heaviest axis of snow. At least it's not showing bizarre 40" amounts. Also, what's with that hole toward CT/MA? Grade: D

Seventh up - GFS

Roidzilla 2021 Model Performance GFS-Verification

What an embarrassing performance less than 12 hours before game time. It reminds me of someone who tried to mix hard liquor after drinking wine for dinner all night. It ended up vomiting all over itself. Fa propia schifo. Grade: F

Eighth up - EURO

Roidzilla 2021 Model Performance EURO-Verification

I think the EURO went out drinking with the GFS the night before. Ok, it was not as bad as the GFS but a superior model like the EURO should demonstrate much better results than this. The northern cut-off of the heaviest axis of snow is surprising. I pulled the Crapola map just to see if that looks any better:

Roidzilla 2021 Model Performance EURO-Crapola-Verification

Nope, not really. Disappointed, honestly. I also feel betrayed. Might need therapy. Grade: D

My overall takeaway is the CMC is better than the GFS/EURO in under 12 hours, and the NAM is by far the superior model in the short range. Then again, this is just one storm. Maybe I should do these more often and see if there is a trend.


_________________
_______________________________________________________________________________________________________

CLICK HERE to view NJ Strong Snowstorm Classifications
Frank_Wx
Frank_Wx
Godzilla Seeker
Godzilla Seeker

Posts : 20002
Reputation : 311
Join date : 2013-01-05
Age : 28
Location : Morris Plain, NJ

http://njstrongweather.blogspot.com/

brownie, essexcountypete and billg315 like this post

Back to top Go down

Roidzilla 2021 Model Performance Empty Re: Roidzilla 2021 Model Performance

Post by aiannone Sun Feb 07, 2021 2:52 pm

@Frank_Wx wrote:I thought this would be a fun exercise and spark some debate. I tried to pull every model's 12z run from Sunday, January 31st and their forecasted snow maps through Tuesday, February 2nd. I realize some areas were already seeing overrunning snow 12z Sunday, but I figured a 12z snapshot on January 31st would still capture majority of the overrunning plus obviously the entire duration of the main event that took place late Sunday all the way into Tuesday.

First up - RGEM

Roidzilla 2021 Model Performance RGEM-Verification

I felt the RGEM was too aggressive. DC to SNJ to CNJ did not get nearly as much as it predicted, and it did not do well in west-central NJ and east-central PA. Its jackpot zone was too expansive, but it got the 'general' jackpot area correct across NW NJ/NEPA. A little underdone in NE NJ. I expected better from a short range model. Grade: C

Second up - 12KM NAM

Roidzilla 2021 Model Performance NAM-Verification

Impressive. It predicted the lower amounts in the southern portions of the region and was also good in outlining the jackpot area. It did under-call central Long Island, but I am not sure any model predicted the subsidence over that part of the island. Grade: A. Bravo, bravo!

Third up - 3KM NAM

Roidzilla 2021 Model Performance NAM-3-KM-Verification

Ehhh it almost looks like it forgot east and west. For real, the small jackpot area it showed in Monmouth County is nearly the same latitude as Allentown, PA which in reality saw close to 2 feet while Monmouth was mostly below 2 feet. NW NJ did better than NE NJ. I don't know, this is some weird ****. Grade: C

Fourth up - HRDPS

Roidzilla 2021 Model Performance HRDPS-Verification

Ma che cazz? I know this is 'Crapola' but I checked and the 10:1 map is very similar to this one. This is the saddest performance I've ever seen. Grade: F

Fifth up - CMC

Roidzilla 2021 Model Performance CMC-Verification

Also on the aggressive side. Especially toward SE NY and SW CT. Also it seemed to under-call north-central NJ area. I don't think it was as bad as the RGEM, though. Grade: B

Sixth up - Icon

Roidzilla 2021 Model Performance Icon-Verification

Similar to the 3KM NAM, seems this model could not pinpoint the heaviest axis of snow. At least it's not showing bizarre 40" amounts. Also, what's with that hole toward CT/MA? Grade: D

Seventh up - GFS

Roidzilla 2021 Model Performance GFS-Verification

What an embarrassing performance less than 12 hours before game time. It reminds me of someone who tried to mix hard liquor after drinking wine for dinner all night. It ended up vomiting all over itself. Fa propia schifo. Grade: F

Eighth up - EURO

Roidzilla 2021 Model Performance EURO-Verification

I think the EURO went out drinking with the GFS the night before. Ok, it was not as bad as the GFS but a superior model like the EURO should demonstrate much better results than this. The northern cut-off of the heaviest axis of snow is surprising. I pulled the Crapola map just to see if that looks any better:

Roidzilla 2021 Model Performance EURO-Crapola-Verification

Nope, not really. Disappointed, honestly. I also feel betrayed. Might need therapy. Grade: D

My overall takeaway is the CMC is better than the GFS/EURO in under 12 hours, and the NAM is by far the superior model in the short range. Then again, this is just one storm. Maybe I should do these more often and see if there is a trend.


Awesome write up Frank. I also feel the NAM has been doing well for many winter storms so far. It was the first model to bring the current storm back.

Secondly, the RPM model has been abysmal. TV mets really shouldnt show it to the public due to how misleading it has been. Had the previous storm and this storm mainly rain for nearly more than half the event which made me hear from people "it looked like mostly rain on the futurecast..." TERRIBLE MODEL
aiannone
aiannone
Advanced Forecaster
Advanced Forecaster

Posts : 4253
Reputation : 89
Join date : 2013-01-07
Location : Saint James, LI (Northwest Suffolk Co.)

Back to top Go down

Roidzilla 2021 Model Performance Empty Re: Roidzilla 2021 Model Performance

Post by billg315 Sun Feb 07, 2021 3:55 pm

Great job Frank. I didn’t really do an analysis like this but I remember my off the cuff feeling being that the NAM did pretty good - which your analysis confirms. But also is ironic -and cautionary, because the first couple NAM runs when it first came in range were waaaay off. Basically had no significant snow N&W of NJ and the jackpot too far SE. Then it corrected overnight Saturday. I guess the lesson there is, the NAM is good - but give it some time (a couple runs) to sort itself out once in range.
billg315
billg315
Advanced Forecaster - Mod
Advanced Forecaster - Mod

Posts : 3626
Reputation : 182
Join date : 2015-01-24
Age : 47
Location : Flemington, NJ

Back to top Go down

Roidzilla 2021 Model Performance Empty Re: Roidzilla 2021 Model Performance

Post by rb924119 Sun Feb 07, 2021 4:28 pm

It has been my anecdotal opinion that ever since the NAM’s last upgrade several years ago now, it has dominated every other piece of guidance within it’s temporal range. THAT SAID, and this gets to Bill’s point, THIS SEASON the NAM has consistently been too progressive in its extended range, only to then correct significantly further northwest inside of 48 hours, largely leading the charge when it does so (and in complete contrast to its ‘usual’ bias of over-amplification). WITH THAT SAID, this anecdotal evidence/opinion cannot be taken as invariably true from year to year or even season to season, as often times each season features a series of seemingly different model biases. Why this is or what causes it, I have no clue lol

This is definitely a fun exercise, though, Frank!!

rb924119
Meteorologist
Meteorologist

Posts : 5354
Reputation : 190
Join date : 2013-02-06
Age : 29
Location : Parents: Greentown, Pa/Me: L.B.I., NJ

Back to top Go down

Roidzilla 2021 Model Performance Empty Re: Roidzilla 2021 Model Performance

Post by Sponsored content


Sponsored content


Back to top Go down

Back to top


 
Permissions in this forum:
You cannot reply to topics in this forum