I've been in construction in one form or another (residential, commercial, utility, infrastructure) since I graduated college. Actually since middle school if we are getting technical, as I worked summers for my dad.
That was great intuition you had. Not going to lie, home owners are a pain in the ass and you don't want them around for numerous reasons from safety, bizarre request, to a million questions that slow you down, etc etc. With that said, they are the client and any trade should accommodate any reasonable request.
If they would not allow you to either see their work or talk to previous customers, that is a huge red flag. There are exceptions like if their current build was for a Paul Skenes type of celebrity, I could see why they would not want you seeing that job from a confidential stand point. But for the most part, they should be transparent with their work.
Great job following your gut.