It should be pretty fine. The bigger chunks can damage the goby's gills when he sifts the sand. That being said, my pistol loved the little pieces (like marble-sized) of live rock rubble I threw in there for him.
I have a yellow watchman goby paired with a tiger pistol shrimp with two clowns. The shrimp has never gone after the two clowns, he and the goby just hang out digging burrows under the rocks all day. The are a comical pair.