On the OG circuit, the delay pot goes to digital ground of the PT2399. This is supposed to be connected to analog ground of the PT2399 internally via a 10R resistor. But, often people run into it just not working that way. So, jumpering the two pins together usually fixes the problem.
Now, that may have nothing to do with your repeat problem. That could be a dodgy pot or something wrong in the circuitry of the feedback path such as a short to ground, bad solder joint etc.
As far as modulation the first suggestion may fix that. It's also possible to get a delay to modulate on its own by simply handling the circuit board, believe it or not. I've seen this a few times with analog delays. If you shake it around a bit the delay pot wiggles a bit and causes the repeats to change pitch slightly like you hear in modulation.