Traps are hard to spot via Divination, them knowing the layout may lead them to false confidence and forget that traps are a thing.
Also doors, Arcane eyes can't go through doors or similar obstacles they need a gap to go through.
Illusions in general, or just creatures that would naturally be hidden while sitting around in the room such as spiders or snakes, or creatures that appear as other objects, such as a roper, animated statues, oozes, animated plants or even the classic corpses that rise as zombies when disturbed.
Creatures blessed with non-detection do not show up to divination spells, so maybe that room that appeared to have three bandits in, actually has three bandits and two bandit-mages in.
Also, creatures with the ability to see invisible things will be able to notice the arcane eye. Maybe one of the bandit leaders has a lantern of revealing mounted on the wall burning with a blue flame, allowing one of the guards to go "Eer boss there is a little floating eye here, what do I do?" "I don't know, put a bucket on it for now."
Of course, don't always shut down the Arcane Eye. It's a pretty high spell slot to invest into scouting so having it shut down every time will just frustrate them, a player in our group used a Shadow sorcerer who frequently used the darkness spell to take cover in while shooting cantrips or blind enemy archers. Of course soon every encounter either had a few blind-sense enemy, or an enemy caster to dispell magic. Another player had a net-fighter who would disable high-value targets instead of dishing out flat damage, but soon enough every encounter had an extra pair of goblins who seemed to be on exclusive "Net-slashing" duty to defend their boss, even fights that seemed odd to have a pair of lackeys around for.
Our game is just Nat1 is a slight fumble, you lose your reaction and bonus action until your next turn, giving enemies a chance to maneuver around you freely, and maybe costing you an attack.