Graveyards are burial grounds attached to a church. Cemeteries are stand-alone sites.
However, a number of sources provide a distinction between the two. The word cemetery comes from Roman times meaning a burial ground typically not associated with a church. Graveyard dates back to the 18th century and referred specifically to a burial ground that was part of a churchyard.
It is worth noting that the two sources cited above offer conflicting information about which word came first, but agree with respect to the meaning of each word.