Let me start by clearing one thing - poles are not defined by magnetic field. Poles are defined by axis of rotation. Magnetic poles are somewhat independent from geographic poles and are only used as handy approximation, especially when you can't get a hold of more accurate measurement e.g of sun and stars, when the sky is clouded. https://en.wikipedia.org/wiki/Poles_of_astronomical_bodies
As for your actual question - it depends on what you mean by large asteroid. Really big ones (bigger than 400km diameter) end up being nearly spherical due to their own gravity, we recently decided to call them dwarf planets. All dwarf planets rotate mostly regularly so you don't need to invent any new system, just choose a prime meridian, maybe a place of first landing, or the highest mountain, and you're good. https://en.wikipedia.org/wiki/Dwarf_planet
Smaller, irregular asteroids, may not map nicely, especially if their rotation is very irregular, but you can still define the main axis by calculating mass distribution, choose the axis that gives maximum moment of intertia, which for regular bodies overlaps with axis of rotation, choose prime meridian (same as above), and you're good. 196
https://en.wikipedia.org/wiki/Moment_of_inertia
For very small asteroids, don't bother with Earth-like coordinates. If your total area is comparable with big cities like London, Moscow or Paris, do something similar. Define "districts" and "neighbourhoods" with memorable names. Place couple of beacons and plaques to make it clear which is where. Everybody will learn them after living there for couple of weeks.
In conclusion.
For any reasonably spherelike object you can define a North Pole, where the major axis crosses the surface, choose an arbitrary Prime Meridian and plot an Earth-like grid of coordinates by simply projecting an imagined sphere on the actual surface. The asteroid doesn't even have to be very regular. These basic rules will work for all kind of potato shapes, as long as it has mostly positive surface curvature, which will always be the case except for very small asteroids. For those, you don't need a grid because the very small area and irregularity allows you to name all sectors unambiguously and just use those names. 204