Now, if we look at the manga, it's not very clear because the paneling leaves us unsure as to where Aoba is looking, and also as there was more to the conversation that Azuma had with Aoba a day or two before, though Azuma does indeed suspect that she hadn't meant to give him chocolates.
However, the anime quite clearly states which camp it is in on this particular development, what with Aoba's glance, the changed dialogue, and the wavering in Aoba's voice when Akane drops by.
Now I'm even more interested in how Adachi-sensei is going to resolve this!
p.s. This theory by Sackett on Animesuki, that it was actually Aoba giving Koh chocolates on behalf of Wakaba, is actually very interesting...and totally in fitting with her character at this point (IMHO).