It is said that 8051 has an on-chip oscillator but requires external clock to run it. What may be the internal circuit for on-chip oscillator, how it runs after connecting crystal oscillator? Also what is the purpose of external capacitor with crystal oscillator?
An IC can manage quite well without having any external crystal or RC oscillator.
While a TTL circuit can oscillate without any noise. An inverter can be designed to always power up with the output in a fixed state - but if that output is connected to the input, the chip will then see a reason to invert. So toggle the output. Which will again make it see a reason to invert, and toggle the output. The timing here controlled by the delay in the TTL logic.
The crystal is there to define the oscillating frequency, just because a crystal is extremely exact (at least when compared to an RC circuit). Chips with internal RC oscillators can manage well enough for UART communication by having the internal RC components laser-trimmed to better than 0.5% precision.
An oscillator obviously need an amplitude larger than 1. But depending on design, it needn't be close to 1. The TTL inverter have an amplification that is way much more than 1. It's just that the amplification needs to happen in a bandwidth span where you the phase change between in and out is enough.
Back to noise - for lots of crystal oscillators, noise is the only way that makes the crystal start to spin up the oscillation. And that is why some processors may fail badly to start the oscillator with some crystals or with some power-up speeds. So with a slow power-up, the drive pin can slowly ramp op the signal but there isn't enough signal seen on the oscillator input pin to magnify and feed back.
Another issue is of course that a crystal may oscillate at the fundamental frequency - but can also have overtone oscillation. So burden capacitors and filtering of the input signal can result in a circuit that always locks onto the same frequency. Or sometimes starts with fundamental and sometimes overtone frequency.