I am using 8051 based device, I got 64K external Flash as code storage, but I found that when my code size approch 64K size limit the program will becomes very unstable and hang abnormally. Does any one encounter this kinds of problem, any clues ?
"Does any one encounter this kinds of problem" Yes - they're called BUGS! The more code you have, the more likely it is that there'll be bugs in it! "any clues?" Get out the Debugger! First, you need to determine what's actually going wrong; then find out why it's going wrong; then you can fix it. Without more details from you, there's little more that can be said.
"they're called BUGS!" Note that it's not limited to just software bugs - you might have a fault in your memory hardware that only happens to show up when you start exercising a certain (high) address range...