On the locus of limit Hodge classes