Video playback starting before video has properly buffered causing it to restart

[Tam]

When streaming a video, the playback starts then a quarter of a second later restarts from the beginning again. Sometimes the video and audio will play before restarting, but usually it's just the audio. Is there anything I can do to force the player to wait for a larger buffer before starting playback?