Basically, the boy (Yoshida) misunderstood that the girl (Hayama) can't handle looking directly to people's eyes.