Hi Charlie,
Thanks for your question!
If you navigate to the speaker library (admin > events > speakers) then hover over the dropdown next to the speaker you want to remove, you will see a delete option.
Hope that helps!
------------------------------
Sydney Langer
CSA
Cvent
------------------------------