Embroidery is not something I have done. But it seems like you need to have a 'well-made' fabric if you are counting warp and weft threads... if the threads are your grid, then loosely-woven fabric, small pieces that can be easily racked or twisted, or fabric with large threads (canvas-like cottons), could all be an issue.
It would seem that embroidery should be done on a piece of fabric with finished edges. Sew a basic rolled scarf seam around the edge, and that will ensure the threads don't slip and the fabric doesn't warp on the bias, and throw your grid off.
Then, when done, cut off the seam by trimming the embroidered piece to shape/size.
So, in short, to answer your question, I'd say both things are an issue with this specific piece. And the problem made worse because the size of the pieces were rather small, had no seam allowance, and working the piece with the embroidery needles just caused the threads to loosen and fray with too much handling.
Does that help?