yes, bypassing that circuit would be ideal, if it exists inside the lamp. I would actually advise you open it up to check if it's relatively easy before probing around.
As for LEDs, I'd suggest you go with some high intensity power LEDs like 3W LEDs.
Now to connect up your LEDs, you've got many many options of circuits to safely drive your LEDs, two of them are worthy of note. Firstly the simplest, is to add a small resistor in there to limit the current of the LED. In some instances you can actually get away with not using a resistor if your battery voltage matches your LED's nominal forward voltage well. If you use Alkaline D cells, then your battery voltage ranges between 6V and 9V depending on charge, in which case you should use two 3W LEDs in series, and a 2W rated 2.7 ohm resistor. If you use ONLY NiMH cells, then you can in theory get away without a resistor when using two 3W LEDs in series, though to be on the safe side, you might want to add a <1 ohm resistor in there, you can use 1/4W rated resistors in this case since you won't be expecting to dissipate much power in the resistor.
Note: you could also double up the circuit if you wanted more light output, simply by adding in a second chain of two LEDs (plus the resistor) in parallel with the first.
The problem with using a resistor is two-fold, firstly you're loosing power in the resistor, and secondly because there's not actual regulation of current (the resistor is there just to limit the current), it means that the lamp dims as your battery runs down. This may be a good thing - it'll remind you when to change the battery, and maybe you might even get sufficient light on nearly empty batteries.
The other choices involve more complex LED driver circuits, though you may be able to buy a driver module off the shelf.
My suggestion however, is to go with the resistor option, and try it out with nearly-flat batteries, and see if the light intensity is still adequate, if not then go with the regulator.