A new federal survey throws shade on those who fish New Jersey coastal waters.
Touted by one coastal newspaper as a study to assist NOAA Fisheries learn “how anglers get information,” an agency report released last fall may simply lead to more questions about how NOAA Fisheries actually gets their information!
A 2020 survey of saltwater anglers found the most used, most trusted sources for recreational fishing information related to management and data collection, in order of importance, are friends, family, state agency websites/guides, bait and tackle shops, federal agency websites/guides, and newspapers and fishing magazines. The report released by NOAA Fisheries on October 15 also found those at the very bottom of list in terms of use and trust by anglers are environmental advocacy groups, social media and online message boards.
NOAA Fisheries conducted the mail survey of 10,000 randomly sampled licensed/registered saltwater recreational anglers on the Atlantic and Gulf coasts from Maine to Mississippi last summer. According to the federal agency in charge of our coastal fisheries, the study is the first phase of an ongoing social network analysis to help identify relationships, networks, channels, and information flow within the recreational fishing community by examining how anglers gather, share, and evaluate information on topics related to fisheries data collection.
While the press release accompanying the study was titled New Study of Recreational Anglers Identifies Most Trusted Information Sources, the irony in the data may prove that “trust” is a two-way street that NOAA Fisheries still finds difficulty in navigating. A subsequent investigation by The Fisherman Magazine in reviewing the NOAA Fisheries findings uncovered the fact that saltwater anglers from Pennsylvania were purposely excluded from the survey, while New Jersey as a whole was grossly under-sampled at rate of nearly one-sixth that of neighboring New York and half that of other Mid-Atlantic States including Maryland and Virginia.
The Data Evolution
This year marks 15 years since the National Academy of Sciences (Academy) performed a comprehensive 2006 review of the Marine Recreational Fishing Statistics Survey (MRFSS), finding the “methods used in the MRFSS for sampling the universe of anglers and for determining their catch and effort is inadequate.” The Academy review came at the request of members of Congress with ample pressure from Rep. Frank Pallone of Monmouth County; then ranking Democrat on the House Fisheries Conservation, Wildlife, and Oceans Subcommittee on Capitol Hill, it was Rep. Pallone who called for a 2004 Subcommittee hearing on fisheries data collection methods.
“One of the main problems with the survey is that it is largely run by federal government contractors, not the states,” Pallone said after the hearing, adding “It is obvious from today’s testimony that giving a larger portion of control over to the states will create better confidence in the data.”
The political pressure ultimately led to the comprehensive MRFSS review by the Academy, a private, nonprofit organization comprised of the nation’s leading researchers considered the most unbiased peer-review panel of experts in the country. Following release of their MRFSS review – one researcher in particular tabbed MRFSS “fatally flawed” – the Academy recommended that data collection methods be “completely redesigned to improve its effectiveness and appropriateness of sampling and estimation procedures, its applicability to various kinds of management decisions, and its usefulness for social and economic analysis.”
Those findings were ultimately integrated into the Magnuson Stevens Fisheries Management and Conservation Act (Magnuson) of 2007, with Congress calling for a complete overhaul of the system, including incorporation of a national angler database for survey purposes. That’s the reason why all saltwater anglers in New Jersey are required to register to fish every year (saltwaterregistry.nj.gov) so survey folks have that contact information at their fingertips. Over the next 10 years, a team of scientists, statisticians, bureaucrats and private contractors worked together to launch the new Marine Recreational Information Program, or MRIP, a replacement system for MRFSS that’s been theoretically built on top of the national saltwater angler registry
In January 2017, the Academy met again, this time to review MRIP, releasing yet another independent assessment of the recreational data collection methodologies. While they found “impressive progress” in the MRIP data collection and analysis, the report highlighted the need to further emphasize “interactive (two-way) communication” with the recreational fishing community, due to wide variations in angler trust in the program.
This historic timeline of MRFSS to MRIP is important for understanding this “social network analysis” project developed by NOAA Fisheries staff and the private contractors at ECS. Essentially, the National Academies of Sciences reviewed MRIP’s five-year strategic plan (2017-2022) and assigned certain tasks for NOAA Fisheries and the ECS private contractors; whether or not those tasks are being completed with efficiency is open to debate, especially when looking at the latest sampling of tristate anglers.
Keystone Crush
When the final Analysis of Marine Recreational Angler Information Gathering and Sharing Habits and Opinions Regarding Fisheries Management and Data Collection report was released on October 15, the 111-page study was dated July 30, 2020. After significant back and forth between The Fisherman and researchers responsible for the report, the study has since been updated.
“After reviewing your questions, our team agreed that the description of the allocation process should be revised to make it more clear,” said Kate Goggin, Public Affairs Specialist at NOAA Fisheries, adding “NOAA Fisheries will publish a revised report within the next few days that includes this updated text.”
So what exactly was updated? First, the original report described how the study sampled all coastal anglers without any notation that some – like those living in Pennsylvania – were completely eliminated from the sample pool. “We sampled anglers in coastal states where the Fishing Effort Survey (FES) is conducted,” Goggin explaining, adding that Pennsylvania anglers were excluded from the survey because the fishing effort surveys themselves are conducted in-person outside of Pennsylvania. FES is one of the two parts used in the overall MRIP sampling to compile recreational harvest rates on a yearly basis, the other part being the angler registry database which NOAA uses to mail out participation surveys (it should be noted coincidentally that the author had never been sent a participation survey until three weeks into conversations with NOAA Fisheries related to this story).
Goggin explained that while participation estimates were used for initial planning purposes of this particular study, the project team instead used a sample frame of resident licensed anglers on the NOAA Fisheries National Saltwater Angler Registry to choose who received survey forms. “The decision to sample the FES states was made for efficiency, given the resources available for the mail survey,” Goggin said, adding “Out-of-state anglers clearly play an important role in recreational fisheries, however, we did not include them in our study due to cost constraints.”
Goggin said the survey was designed to be representative of the Mid-Atlantic Fishery Management Council (Council) region as a whole, and despite the fact that Pennsylvania has a seat on the Council and a voice in Mid-Atlantic coastal fisheries as required under Magnuson, researchers opted to exclude Pennsylvania residents.
In a November 2 conference call with government staffers and researchers, John Foster, Branch Chief for the Office of Science and Technology’s Recreational Fisheries Statistics Branch at NOAA said “It wasn’t intentional to disenfranchise or slight Pennsylvania resident saltwater anglers, that was not the intention at all.” Foster said the purpose of the study was to identify communication pathways, information sources, and frequencies of use for those information sources by saltwater anglers, but explained that limiting this particular survey to the FES was necessary given the resources.
“I understand there would be an interest to get this same information from Pennsylvania anglers and potentially from all anglers that don’t reside in the immediate, adjacent coastal states,” Foster said, adding “But again, decisions had to be made about where to draw the line in terms of the resources we had to allocate to the study.”
Checking Off Boxes
In addition to understanding where coastal anglers get their fisheries information and regulatory news – as well angler trust levels associated with different regulatory sources of information and data collection – another primary objective of this study ironically was to gauge the angling public’s understanding of and confidence in fisheries management, data collection, and data analysis. It was also targeted to collect angler opinions and beliefs regarding the current state of saltwater recreational fisheries management and data collection, aimed at helping the agency “tailor outreach efforts to topical areas where anglers trust in the management and data collection process is weakest.”
So if the Academy’s 2017 report recommendations resulted in this study as a way for NOAA Fisheries to improve two-way communication and foster trust with anglers, the agency seemingly checked off boxes for a task while working in pencil with an eraser. Prior to October 23 when the paper itself was modified by NOAA, researchers described the way each state was represented in total sample allocations by referring to MRFSS participation estimates from 2016.
“The weighted allocation is the number of units on the frame which was determined by a MRFSS 2016 participation query by ECS and NOAA Fisheries MRIP Staff. The sample allocations were designed to match the MCR and state-level populations of the same MRFSS 2016 participation query.”
Based on the 2016 MRFSS numbers, NOAA’s Mid-Atlantic regional data shows New York has having 808,544 resident anglers, followed by New Jersey with 538,466, Virginia with 480,037, Maryland at 476,384, and Delaware with 103,691.
However, the sampling rates used by NOAA Fisheries for this particular study ranked states in a different order of importance (with percentage of sampling rates listed in parenthesis), with New York ranked first with 587,848 registered anglers (a 13.2% sampling rate), followed by Virginia with 233,533 (5.8%), Maryland with 223,107 (5.5%), New Jersey coming in at 110,490 (2.75%) and Delaware ranked last with 51,444 registered recreational anglers (2.28%).
When queried as to why New Jersey was grossly under-sampled at a rate of 2.75% despite MRFSS 2016 participation data showing otherwise, NOAA staffers said the allocation formula had been changed during the study.
“While the participation estimates were used for initial planning purposes, the project team decided to use a sample frame of resident licensed anglers on NOAA Fisheries National Saltwater Angler Registry (NSAR),” Goggin said, explaining how sample allocations were based on the number of resident licensed anglers in each state and adjusted to account for differential response rates among states observed in the FES from 2016-2018.
Response rates in this particular study were rather shocking and could show why the federal agency needs to sample New Yorkers at a higher rate; according to the study data, New Jersey saltwater anglers responded to the survey at 49.09% response rate (the highest rate of response among all Atlantic and Gulf Coast states) while New York was dead last with a 23.49% rate of response.
Damned Lies & Statistics
Typically when a scientific report or study is altered after the fact, it includes footnotes which describe the change and where that modification was made. In this case, researchers simply removed the previous references and replaced it with new ones, overwriting previous sources. In both instances with this particular government study – the reference to instate anglers and the change to the angler registry over previous MRFSS estimates – the references were simply rewritten.
“Anglers with licenses valid in 2018 were included on the sample frame,” Goggin said later by email, after the fact. Which only begs the question; does New York really have 587,848 registered instate New York anglers in the angler registry database in 2018 as NOAA Fisheries contends? According to Jim Gilmore, the director of marine resources at the Department of Environmental Conservation in New York, there were a total of 407,342 marine anglers in their registry database in 2018, which ostensibly includes both in-state and out of state anglers. “A steady increase although still significantly lower than federal estimates on total number of anglers,” Gilmore astutely noted when providing the actual numbers.
So, NOAA Fisheries’ is off by at least 180,000 saltwater anglers; when you consider the number of people who register to fish New York marine waters from out of state who theoretically should be excluded from the survey (as in Pennsylvania), that glaring error should only grow by tens of thousands of individual anglers. While even casual observers can see the sampling rate inconsistencies with this particular study, NOAA Fisheries was quick to point out that this is in no way indicative of MRIP itself.
This story began at 2 p.m. on October 15, 2020 after sitting through an hour-long webinar – Marine Recreational Information Gathering Habits and Opinions on Fisheries Management and Data Collection – presented by NOAA Fisheries. The NOAA Fisheries press release touting the report was issued a little after 4 p.m.When reaching out to NOAA Fisheries on Friday, October 16 with questions related to the survey, it would take another week before receiving a response from Kate Goggin, Public Affairs Specialist at NOAA Fisheries. “More samples were drawn for NY than NJ because NY had a higher number of resident licenced (sic) anglers on the NSAR,” she said.Goggin went on to explain that consideration was also given to the fact that New Jersey anglers show a higher response rate with regard to sampling when compared to New York, particularly with regard to the Fishing Effort Survey (FES). “The three-year average return rate on the FES for NJ was 49.82% and the NY average return rate was 37.12%,” Goggin said, adding “Although the NY FES response rates weren’t as high as NJ, NY still had more than five times the number of licensed resident anglers, which warranted the larger sample.” According to the New Jersey Division of Fish and Wildlife, there were over 158,000 saltwater anglers registered to fish New Jersey waters in both 2017 and 2018.
As for the incredible disparity in survey response rates between saltwater anglers from New Jersey (49.09%) and New York (23.49%), Goggin seems to acknowledge that the higher participation rate in sampling was taken into account for this survey. That leads me to wonder how that affects MRIP sampling throughout the region.
As required under the Modernizing Recreational Fisheries Management Act of 2018, NOAA Fisheries is expected to submit a biennial report to Congress detailing its progress toward the recommendations issued by the National Academies of Sciences in 2017. The question is, how will Congress actually respond? – JH
“The goal was not, in this study, to estimate the total number of participants in a given state or to use it to somehow allocate specific resources for monitoring or for policy making or management purposes,” Foster said, stressing that this particular research was not the actual MRIP survey, but an internal project for NOAA Fisheries.
“It was really just to identify pathways of information or sources of information that anglers use and how frequently they use them for our own agency’s planning purposes for communications,” Foster explained of the purpose of the study.
Yet according to the paper itself, the researchers noted “This research project was developed and implemented to inform MRIP’s efforts to improve stakeholder engagement and foster two-way communication between MRIP and recreational anglers.”
Sixteen years later, that song about two-way communication and outreach still sounds eerily similar to some. We reached out to Congressman Pallone in November to share our findings, and you could practically hear the head-shaking over the phone.
“This is the same problem we’ve seen before,” said Rep. Pallone, who added “It’s important that the MRIP study accurately reflect the Mid-Atlantic’s recreational fishing sector. I’m going to work with NOAA and request more information to ensure their analysis captures a precise assessment of our region’s fishing industry.”
