Issues with Custom Nameservers and DNS on Namecheap for WordPress Migration

I’m migrating my site to WordPress, which requires pointing to their nameservers. In Namecheap, it seems I need to use Custom DNS rather than BasicDNS to do this. However, when I switch to Custom DNS, all my records (A, CNAME, MX, etc.) disappear, and I’m told to manage them through cPanel, which seems to be a paid option, or by using their EasyWP service.

From what I’ve read, it appears these records don’t transfer when switching to the custom option. Is there a way to point to the WordPress servers without having to pay for anything additional? I thought changing nameservers would be a straightforward process. Any advice?