NOAA Bombshell! Angler Effort Surveys Still Flawed - The Fisherman

NOAA Bombshell! Angler Effort Surveys Still Flawed

A good striper caught and released by the author’s dad in 2022 aboard Chuck Many’s Tyman, but was overall angling success last season really as robust as the data suggests?

Government officials find “reporting errors and illogical responses.”

In a conference call with recreational fishing stakeholders on August 7, 2023, NOAA Fisheries admitted that their new Fishing Effort Survey (FES) used to estimate recreational saltwater participation along the Atlantic Coast and Gulf of Mexico is likely overestimating angler effort by as much as 30 to 40 percent.

Launched in 2015 as part of the Marine Recreational Information Program (MRIP) which replaced the earlier harvest/effort survey known as MRFSS (Marine Recreational Fishing Statistical Survey), the FES uses mail surveys administered for six independent two-month reference waves.  According to an FES 2018 Annual Report, these surveys utilize address-based samples within coastal states to collect information about recreational saltwater fishing activity, with “sample frames” consisting of all residential addresses within coastal states serviced by the U.S. Postal Service.

MRFSS was ultimately replaced by MRIP following a 2006 study by the National Academy of Sciences and its National Research Council (NRC) that found the old survey method to have “serious flaws in design or implementation and use inadequate analysis methods that need to be addressed immediately.” Pat Sullivan, the NRC committee chair at the time referred to MRFSS specifically as “fatally flawed” and the findings ultimately led to a mandate by Congress that a new survey methodology be implemented by NOAA Fisheries.

The latest acknowledgement by NOAA Fisheries and its Office of Science and Technology comes by way of a May, 2023 report called Evaluating Measurement Error in the MRIP Fishing Effort Survey using several pilot studies and data analyses to evaluate non-sampling errors in the survey design.  Results of the ongoing series of studies indicate that the order in which certain questions are presented in the surveys mailed to anglers “may not be optimal in terms of reducing ‘telescoping error’” which is a statistical way of saying “over-reporting.”  NOAA Fisheries said anglers responding to the mail-in survey may not be reviewing the entire questionnaire and therefore are improperly answering certain “bounding questions” (12-month shore and boat fishing) that would help to reduce telescoping error.

According to Evan Howell, Director of NOAA’s Office of Science and Technology, the federal fisheries service collects private recreational fishing trip information for each resident of a responding household using a list of residential addresses from the U.S. Post Office which is matched to the National Saltwater Angler Registry derived from state licenses and registrations.  “The FES is part of a group of surveys NOAA Fisheries uses to estimate total recreational catch,” Howell said, explaining how the ordering of questions contained within those surveys may lead to significant bias.  “The results of this study suggest that the order of the questions in this survey may lead to over-estimation of fishing effort,” Howell stated.

Explaining the process to stakeholders, Howell said the current FES questionnaire first asks people to report their fishing activities over a 2-month period, and then a 12-month period.  However, the pilot study report from May of 2023 reversed the order of those two questions, with rather significant results.  “This pilot study concluded that the revised question order resulted in fewer observed reporting errors and illogical responses, where there was more fishing done in the 2-month period than in a 12-month period, and the resulting effort estimates were generally 30 to 40 percent lower for shore and private boat modes than produced for the current FES design,” Howell stated.

Howell went on to explain that the federal agency is planning a larger-scale follow-up study to gain a clearer understanding of the differences in effort estimates between the current design and a revised version that changes both the question order and increases the frequency of sampling, but he said it’s not yet clear if they’ll achieve the same results.

“The follow-up study will be conducted over the full course of 2024 with the revised design administered alongside the current design,” Howell said, explaining how monthly sampling will produce more frequent estimates, while a shorter respondent recall period could also minimize reporting errors.  “Lowering the reporting error rate should ultimately produce more accurate data and resulting estimates of effort,” he said.

While private anglers are surveyed randomly by U.S. mail, those fishing aboard federally inspected and licensed head boats have their catches registered by the captain and crew through the electronic vessel trip reporting system.

What About Stripers?

In a surprise vote back on May 2 by the Atlantic States Marine Fisheries Commission (ASMFC) and its Atlantic Striped Bass Management Board (Board), fisheries managers directed all ocean states to modify recreational striped bass regulations to adopt a one fish bag and a 28- to 31-inch size limit.  In an official press release issued the following day, ASMFC’s Tina Berger said the emergency order to enforce the new 31-inch maximum size coastal limit for 2023 was made to reduce the directed harvest of the strong 2015 year class based on the data coming out of MRIP and the FES.

“This action responds to the unprecedented magnitude of 2022 recreational harvest, which is nearly double that of 2021,” Berger said in the release, explaining how angling effort in 2022 based on MRIP jeopardized the stock rebuilding projections for striped bass.  As reported in our June monthly edition (Playing The Slot: ASMFC Declares Striper Emergency), ASMFC made their emergency declaration based on MRIP data for 2022 that showed a 40-percent overall increase in recreational removals of striped bass as compared to the prior year (roughly the same effort percentage differential captured in the FES pilot study).

Essentially, ASMFC’s bombshell vote in the spring was based on recreational harvest surveys now being called into question by the federal fisheries agency.  A final vote in favor of the Massachusetts-led emergency action was approved by a 15-1 vote, with the sole opposition coming from the state of New Jersey.  Then on August 1, the emergency 28- to 31-inch slot measures were upheld into 2024 through another vote by the ASMFC Striped Bass Management Board that passed 14-2, with both New Jersey and the District of Columbia voting in opposition.

I raised the question during the August 7th conference call as to whether or not this information should’ve been presented to the ASMFC Striped Bass Management Board prior to taking a vote on continuing the coastwide striped bass emergency measures, particularly given how the regulatory change was based heavily on the current FES methodologies now in question.  Howell replied that there’s a “process” that needs to be followed.

“We’re not acting on this information right now because that is part of a process,” he said, noting how staff would like to sit down and complete the testing to make sure that any potential changes to the FES system are based on the best scientific rationale possible.  “But right now we have an indication that we need to continue this testing to make this happen,” Howell added.

In terms of the ASMFC decision and the fact that NOAA Fisheries cast yes votes at the commission to continue the emergency regulations for striped bass, despite knowing about this new effort and harvest announcement to come, Howell said “We have this in hand now, it is probably going to be a part of discussions, I don’t see it going into action any time soon. We still have to use the information that we have.”

“For the time being we will still use the FES that we have while we continue these conversations in parallel of how we want to mitigate based on the results of this pilot study,” he added.

Despite NOAA Fisheries acknowledgement about “reporting errors and illogical responses” in their fishing effort surveys leading to upwards of 40% over-reporting, early indications are that fluke limits for 2024 may be trending downwards.

Fluke, Sea Bass & Porgy?

While debate over striped bass measures versus any scientifically established need (based on new findings) may have been a week too late for argument, NOAA Fisheries brought the pilot study findings to Annapolis, MD for a joint meeting of the ASMFC and Mid-Atlantic Fishery Management Council (MAFMC) where 2024 specifications were to be set for summer flounder, black sea bass and scup (porgy).  Chairman Mike Luisi of Maryland kicked off the August 8th hearing by saying the release of the new findings less than 24 hours prior to the joint ASMFC/MAFMC meeting didn’t allow for any relevant discussion towards 2024 quota management.

“I participated in the meeting and I asked a few questions, I was a little frustrated just with the process given the fact that we had a meeting today and this conversation happened at 12:30 yesterday afternoon,” Luisi said, adding “I knew that there would be people around the table, members of the public and others who would be expressing their concerns regarding how we take the next steps based on our agenda for establishing specifications for species when we’re being told that there’s some error and some bias in the information that we are using for those purposes.”

“I don’t know that now is the time to get into the details about this, about the report that came out yesterday, what I’d like to do is move forward with our agenda, get the specifications on the books and then there will be opportunity in the future for more detailed question and answer with the service (NOAA Fisheries), and that’s going to be warranted in the future,” Luisi said, explaining how he only knew about the details in the findings from what he read that morning at 2:30 a.m.

In their August 7th briefing, NOAA Fisheries explained that the 30 to 40% over-estimation swing was not definitive, and the staffers explained how the pilot study findings were not consistent across all states and all fishing modes, shore versus boat. “The effect was a little bit more pronounced for shore mode than for private boat or rental boat mode,” noted Katherine Papacostas, MRIP Program Management branch chief.

Papacostas used New York’s data as an experimental example that showed similar trends with effort being overestimated by as much as 58% in The Empire State’s private boat mode and 55% for shorebound estimates.

“One point of note is we had smaller sample sizes for the experimental treatments compared to the FES in this study, so the experimental point estimates are generally less precise (i.e., have a bit more sampling error uncertainty) than the FES estimates,” Papacostas added, explaining how NOAA Fisheries plans to increase sampling in follow-up studies in 2024.

“The estimates are lower on effort than what the current design indicates, which is a major difference when you’re using those effort estimates to produce catch estimates that go into assessments and are part of the process for us in establishing quotas both recreationally and commercially,” Luisi said of the 30- to 40-percent discrepancy between the new pilot study and the existing MRIP data.  “It’s not like this bias is understood,” Luisi added, noting how the new findings were not being factored into the data being used by the MAFMC currently while calling this a longer term process.

“The follow-up study will be conducted over the course of 2024 and from what I remember from yesterday’s discussion, it is thought that by 2026 the new information that comes from the study, if there is new information, will be factored in to management by 2026,” Luisi said.

Given that “longer term process” the MAFMC and ASMFC voted jointly on August 8 to reduce the recreational harvest limit on fluke by roughly 40% in 2024, which coincidentally is the same high-end disparity in fishing effort surveys coordinated by NOAA Fisheries.  The good news is that recreational harvest limits for porgy should increase next year, even as black sea bass limits may be cut again in 2024.



“Lock And Load” Spring Stripers On Raritan Bay

This “spot” was “burned” decades ago, and it just keeps on getting better!


Giving Back: Oystering On Delaware Bay

More oysters, cleaner water, and perhaps some better fishing.

Talkin’ Tautog: A Concern For Blackfish?

It's not always how to catch blackfish, but how are the blackfish.