Here are more thoughts:

Another way I would think about it would be, what type of information can I get from a Freedom of Information Act (FOIA) request? For instance, I can get a police picture of a traffic accident, but to comply with FOIA policies as I understand them, the license plates in the picture should be blurred. I may or may not know the identities of the cars involved from other sources, but this type of personal information would be restricted. Similarly, I could as an agency how many people died of a heart attack at my local hospital. But I couldn't ask how many 55 year old Army veteran men with diabetes and born in Rhode Island died at my local hospital - because this would likely be personally identifying. I can also ask the IRS about the average income in my area, but not the average income of residents of a single house. Similarly, I could ask a the firearms agency how many guns are permitted to addresses in a certain area (note, there may be legal ownership that is not registered, depending on many factors I do not know or understand). So long as the area is big enough that the information does not result in personally identifying information, I think I would get the information.

In all four, some amount of information is not included in the FOIA or government records request for confidentiality. It seems to me that the idea is that the government has good reason to have the information, but it isn't clear whether or not it should be legal to distribute the info.

For example, in the health case, many people have access to health records. There may be good public interest reasons to release more information than we are legally allowed to (for instance, it may be a public benefit for some nurse to sneak out health information about politicians like Trump or Fetterman). However, doing so would be a crime, and the individuals would rightly be subject to penalty (for an example of this, consider what happened to Chelsea Manning, for better or worse she thought she was doing the right thing although she knew she was breaking the law). The reason I think they would rightly be subject to penalty is because the law cannot possibly be nuanced enough to determine when information should or should not be withheld. Even something less blunt than the law, such as journalistic ethics around exposing confidential sources, often is not nuanced enough and sources get to keep confidentiality even when the public benefit of exposure is high.

So, in your case, you don't really know if knowing this information would be useful; you don't actually know if your neighbor might commit a gun crime. But you could believe you are justified in breaking into your neighbors house and finding out if there is a gun. You could also bribe the firearms permitting agency into letting you see the information. Or you could get yourself hired by an agency that has access to the firearms agency's database. You could eventually feel vindicated in your search if you somehow used that information to prevent a crime in the future (I don't know exactly how this would work, maybe it wouldn't). But we currently think you would have committed a crime.

As with the Manning case, a future governor or president may later pardon you or commute your sentence in recognition of the failure of legal nuance in properly handling your case. Or they may not.

So, while it currently is illegal, circumventing the law is always possible (ie civil disobedience). If you did circumvent the law, you may be vindicated by the writers of history or even by society or a future politician. Or you may not be so vindicated and just be viewed as a creepy busybody.