Hi,
I am using Keil v5 (RTX5) with the EVBK-IMXRT1050 board.
I have successfully sent an email using the example code given through the pack installer, SMTP_client. Now I want to send a secure email, SMTPS, using the same example but adding mbedTLS. I followed the instructions given from Keil, but have not been able to get a secure email to send. www.keil.com/.../use_secure_components.html
Below is my event recorder when trying to send out a SMTPS using mbedTLS, everything works fine until I reach line 41 where TLS should begin. The socket is closed right when TLS should begin and I am not sure why. Wireshark matched the event recorder.
If anyone has had this issue or have any tips on getting around this error, please reply. Thanks.
Event recorder: Event, Time (sec), Component, Event Property, Value 0, 5.45808617, EvCtrl, EventRecorderInitialize, Restart Count = 3 1, 5.45812999, EvCtrl, EventRecorderStart, 2, 5.45863391, Net_SYS, InitSystem, ver=7.10.0 3, 5.45974125, Net_DHCP, InitClient, vcid=0 opt=0 4, 5.45976630, Net_DHCP, StartClient, 5, 5.45981325, Net_SMTP, InitClient, smtp 6, 5.45986813, Net_SYS, InitComplete, success 7, 5.46012398, Net_SMTP, SendMail, recipients=1 8, 5.46044442, Net_ETH, LinkDownError, no-link 9, 5.66058112, Net_ETH, LinkDownError, no-link 10, 7.66053657, Net_ETH, LinkDownError, no-link 11, 8.36051481, Net_DHCP, ClientState, state=INIT 12, 8.36055019, Net_DHCP, SendDhcpMessage, type=DHCP_DISCOVER bcast=1 13, 8.36076685, Net_DHCP, NextState, state=SELECTING 14, 8.36318817, Net_DHCP, ReceiveFrame, server=10.0.2.17 len=300 15, 8.36321537, Net_DHCP, ClientState, state=SELECTING 16, 8.36322062, Net_DHCP, DhcpOfferReceived, 17, 8.36322724, Net_DHCP, ShowServerId, server_id=10.0.2.17 18, 8.36323507, Net_DHCP, ShowOfferedIpAddress, ip=10.0.2.157 19, 8.36323565, Net_DHCP, SendDhcpMessage, type=DHCP_REQUEST bcast=1 20, 8.36334852, Net_DHCP, NextState, state=REQUESTING 21, 8.36697469, Net_DHCP, ReceiveFrame, server=10.0.2.17 len=300 22, 8.36698434, Net_DHCP, ClientState, state=REQUESTING 23, 8.36699382, Net_DHCP, DhcpAckReceived, 24, 8.36699851, Net_DHCP, ShowAssignedAddress, ip=10.0.2.157 25, 8.36705461, Net_DHCP, ShowLeaseTime, time=171072 26, 8.36705907, Net_DHCP, ShowNetMask, mask=255.255.0.0 27, 8.36706187, Net_DHCP, ShowGatewayAddress, gateway=10.0.2.2 28, 8.36707048, Net_DHCP, ShowDnsServers, pri=8.8.8.8 sec=8.8.4.4 29, 8.36707621, Net_DHCP, NextState, state=BOUND 30, 9.66583049, Net_SMTP, ConnectIp4, ip=10.0.1.2 port=587 31, 9.66751124, Net_SMTP, SocketConnected, sock=1 32, 9.66857944, Net_SMTP, ReceiveFrame, sock=1 len=58 33, 9.66860121, Net_SMTP, ServerReady, 34, 9.66878471, Net_SMTP, SendCommand, command=EHLO evkb-imxrt1050 35, 9.66962207, Net_SMTP, ReceiveFrame, sock=1 len=146 36, 9.66966114, Net_SMTP, EsmtpModeActive, 37, 9.66966166, Net_SMTP, TlsSupportIndicated, 38, 9.66977401, Net_SMTP, SendCommand, command=STARTTLS 39, 9.76064354, Net_SMTP, ReceiveFrame, sock=1 len=30 40, 9.76066607, Net_SMTP, StartTlsAccepted, 41, 9.76085824, Net_SMTP, TlsModeStarted, 42, 9.76109192, Net_SMTP, SocketClosed, sock=1 43, 9.76116411, Net_SMTP, ClientCloseSocket, sock=1 44, 9.86054282, Net_SMTP, ClientDone, cb_event=Error
UPDATE:
After finally getting the mbedTLS debugger to print to the Keil event recorder, I receive an error regarding allocating space.
Looking at the event recorder, after line 51, "ssl_tls.c:7484: alloc(15333 bytes) failed" is printed in hex. I have tried to add more space but in the mbedTLS file, ssl_internal.h, I found:
/* * Check that we obey the standard's message size bounds */
#if MBEDTLS_SSL_MAX_CONTENT_LEN > 16384
#error "Bad configuration - record content too large." #endif
Meaning that if the message size is greater than 16384 bytes, it is too big. The message that should be printing is "Client Hello". Now I am trying to figure out why the message size is bigger than normal. Also on line 39, mbedTLS is saying that the Record Type is Not Supported, I am assuming this is not causing the later issue with TLS and that it involves the use of Ip6. Since Ip6 cannot be used, the connection is switched to Ip4 as shown in line 40.
Event Recorder, with print statements: lines 7-15: MW Network SMTP Client Sending email...
lines 52-57: ssl_tls.c:7484: alloc(15333 bytes) failed
lines 61-63: Send email failed.
Event, Time (sec), Component, Event Property, Value 0, 0.00297065, EvCtrl, EventRecorderInitialize, Restart Count = 9 1, 0.00300803, EvCtrl, EventRecorderStart, 2, 0.00347310, Net_SYS, InitSystem, ver=7.10.0 3, 0.00451524, Net_DHCP, InitClient, vcid=0 opt=0 4, 0.00453281, Net_DHCP, StartClient, 5, 0.00458062, Net_SMTP, InitClient, smtp 6, 0.00464454, Net_SYS, InitComplete, success 7, 0.00479595, STDIO, stdout, 0x4D 0x57 0x20 0x4E 0x65 0x74 0x77 0x6F 8, 0.00481483, STDIO, stdout, 0x72 0x6B 0x20 0x20 0x20 0x20 0x20 0x20 9, 0.00481529, STDIO, stdout, 0x0A 0x00 0x00 0x00 0x00 0x00 0x00 0x00 10, 0.00485145, STDIO, stdout, 0x0D 0x53 0x4D 0x54 0x50 0x20 0x43 0x6C 11, 0.00485261, STDIO, stdout, 0x69 0x65 0x6E 0x74 0x20 0x20 0x20 0x20 12, 0.00485311, STDIO, stdout, 0x20 0x0A 0x00 0x00 0x00 0x00 0x00 0x00 13, 0.00487348, STDIO, stdout, 0x0D 0x53 0x65 0x6E 0x64 0x69 0x6E 0x67 14, 0.00487439, STDIO, stdout, 0x20 0x65 0x6D 0x61 0x69 0x6C 0x2E 0x2E 15, 0.00487490, STDIO, stdout, 0x2E 0x0A 0x00 0x00 0x00 0x00 0x00 0x00 16, 0.00494046, Net_SMTP, SendMail, recipients=1 17, 0.00524577, Net_ETH, LinkDownError, no-link 18, 0.20544790, Net_ETH, LinkDownError, no-link 19, 2.20531747, Net_ETH, LinkDownError, no-link 20, 2.90546651, Net_DHCP, ClientState, state=INIT 21, 2.90549117, Net_DHCP, SendDhcpMessage, type=DHCP_DISCOVER bcast=1 22, 2.90562930, Net_DHCP, NextState, state=SELECTING 23, 2.90780736, Net_DHCP, ReceiveFrame, server=10.0.2.17 len=300 24, 2.90784296, Net_DHCP, ClientState, state=SELECTING 25, 2.90784781, Net_DHCP, DhcpOfferReceived, 26, 2.90785995, Net_DHCP, ShowServerId, server_id=10.0.2.17 27, 2.90787479, Net_DHCP, ShowOfferedIpAddress, ip=10.0.2.157 28, 2.90790023, Net_DHCP, SendDhcpMessage, type=DHCP_REQUEST bcast=1 29, 2.90800321, Net_DHCP, NextState, state=REQUESTING 30, 2.91153045, Net_DHCP, ReceiveFrame, server=10.0.2.17 len=300 31, 2.91153428, Net_DHCP, ClientState, state=REQUESTING 32, 2.91154255, Net_DHCP, DhcpAckReceived, 33, 2.91154700, Net_DHCP, ShowAssignedAddress, ip=10.0.2.157 34, 2.91157338, Net_DHCP, ShowLeaseTime, time=171072 35, 2.91158152, Net_DHCP, ShowNetMask, mask=255.255.0.0 36, 2.91158209, Net_DHCP, ShowGatewayAddress, gateway=10.0.2.2 37, 2.91158262, Net_DHCP, ShowDnsServers, pri=8.8.8.8 sec=8.8.4.4 38, 2.91158964, Net_DHCP, NextState, state=BOUND 39, 4.22416890, Net_DNS, RecordTypeNotSupported, rr_type=TYPE_CNAME 40, 4.22426921, Net_SMTP, ConnectIp4, ip=74.125.141.109 port=587 41, 4.25463296, Net_SMTP, SocketConnected, sock=1 42, 4.30152218, Net_SMTP, ReceiveFrame, sock=1 len=54 43, 4.30153476, Net_SMTP, ServerReady, 44, 4.30167701, Net_SMTP, SendCommand, command=EHLO evkb-imxrt1050 45, 4.33734949, Net_SMTP, ReceiveFrame, sock=1 len=168 46, 4.33736641, Net_SMTP, EsmtpModeActive, 47, 4.33736883, Net_SMTP, TlsSupportIndicated, 48, 4.33747245, Net_SMTP, SendCommand, command=STARTTLS 49, 4.36792064, Net_SMTP, ReceiveFrame, sock=1 len=30 50, 4.36792558, Net_SMTP, StartTlsAccepted, 51, 4.36806210, Net_SMTP, TlsModeStarted, 52, 4.36833193, STDIO, stdout, 0x0D 0x73 0x73 0x6C 0x5F 0x74 0x6C 0x73 53, 4.36833566, STDIO, stdout, 0x2E 0x63 0x3A 0x37 0x34 0x38 0x34 0x3A 54, 4.36833644, STDIO, stdout, 0x20 0x61 0x6C 0x6C 0x6F 0x63 0x28 0x31 55, 4.36833722, STDIO, stdout, 0x35 0x33 0x33 0x33 0x20 0x62 0x79 0x74 56, 4.36833801, STDIO, stdout, 0x65 0x73 0x29 0x20 0x66 0x61 0x69 0x6C 57, 4.36833856, STDIO, stdout, 0x65 0x64 0x0A 0x00 0x00 0x00 0x00 0x00 58, 4.36852025, Net_SMTP, SocketClosed, sock=1 59, 4.36862414, Net_SMTP, ClientCloseSocket, sock=1 60, 4.40548247, Net_SMTP, ClientDone, cb_event=Error 61, 4.40555172, STDIO, stdout, 0x53 0x65 0x6E 0x64 0x20 0x65 0x6D 0x61 62, 4.40555268, STDIO, stdout, 0x69 0x6C 0x20 0x66 0x61 0x69 0x6C 0x65 63, 4.40555328, STDIO, stdout, 0x64 0x2E 0x0A 0x00 0x00 0x00 0x00 0x00
It looks like your heap size is too small and the malloc fails. First try to increase the size of the heap so that the malloc will no longer fail.
The "mbedTLS_config.h" preconfigured for SMTP client does however limit the content size to 8k, which is usually sufficient:
#define MBEDTLS_SSL_MAX_CONTENT_LEN 8192
mbedTLS allocates two buffers MBEDTLS_SSL_IN_CONTENT_LEN and MBEDTLS_SSL_OUT_CONTENT_LEN per session. If you reduce max content length by 8k, you save 16k of a heap.
The buffer must be capable of accepting the entire chain of certificates, that is exchanged during the handshake. So if your SMTPS server has set up a certificate chain that is larger, 8k may not be enough. You can, however, do a little experiment with the sizes. Because this is an embedded application, we always face memory limitations.
The size of the message you want to send in an email and the max content length are not related. You can send emails that have hundreds of kilobytes.
uVision debug hint:
STDIO - stdout messages that are in the "Event Recorder" window in hex, are in the text in the second window "Debug (printf) Viewer". When debugging, you need to open both windows.
Hi Franc,
Thanks for the help, I finally got the email to go through using SMTPS. For the most part, I had to change the configuration file and increase the heap size. I also increased the MBEDTLS_SSL_MAX_CONTENT_LEN which fixed my last issue of mbedTLs not sending the TLS "Client Hello".
Another thing I wanted to do is move this SMTPS code over to an older board, lpc3250 Arm9, which is not supported by Keilv5. The Arm9 project is a bare metal environment with no operating system. Currently, the Arm9 application does have a bare metal SMTP protocol hand written, but I want to switch it to SMTPS. I am working on the lpc3250 processor using Keilv4 whcih does not include all the new software packs and run-time environment.
My questions are: 1. Will this mbedTLS SMTPS application work on a bare metal environment without a RTOS? 2. Does mbedTLS require an outside API, such as Windows API, to run TLS? 3. Is it possible to move over all the source code to the Arm9 Keilv4 project and get it to work with mbedTLS SMTPS? 4. I read that some TLS certificates require a date verification and my Arm9 embedded system does not keep track of the current date and time, is this involved in SMTPS?
-Kamal
Also,
I can send the mbed SMTPS TLS1.2 using ports 587 and 25 successfully, is that how it is supposed to work?
My mbedTLS SMTPS program does not work every single run. Sometimes the program will hang or fail. Why is the program not running the same every time? Most of the time when it does hang, it will hang right after:
39, 4.21519186, Net_SMTP, ConnectIp4, ip=10.0.1.2 port=587
Even in the successful email cases, I sometimes receive events like:
19, 2.21033839, Net_ETH, LinkDownError, no-link
39, 4.21497440, Net_DNS, RecordTypeNotSupported, rr_type=TYPE_CNAME
41, 4.21587020, Net_TCP, FrameUnrecognised, sock=1
42, 4.89372711, Net_TCP, NoRetriesLeft, sock=1
43, 4.89384738, Net_SMTP, SocketAborted, sock=1
One example of a failed SMTPS email Keil event recorder:
Event, Time (sec), Component, Event Property, Value 0, 0.00391493, EvCtrl, EventRecorderInitialize, Restart Count = 2 1, 0.00393650, EvCtrl, EventRecorderStart, 2, 0.00445639, Net_SYS, InitSystem, ver=7.10.0 3, 0.00548874, Net_DHCP, InitClient, vcid=0 opt=0 4, 0.00549881, Net_DHCP, StartClient, 5, 0.00554865, Net_SMTP, InitClient, smtp 6, 0.00562303, Net_SYS, InitComplete, success 7, 0.00579318, STDIO, stdout, 0x4D 0x57 0x20 0x4E 0x65 0x74 0x77 0x6F 8, 0.00582144, STDIO, stdout, 0x72 0x6B 0x20 0x20 0x20 0x20 0x20 0x20 9, 0.00582199, STDIO, stdout, 0x0A 0x00 0x00 0x00 0x00 0x00 0x00 0x00 10, 0.00587034, STDIO, stdout, 0x0D 0x53 0x4D 0x54 0x50 0x20 0x43 0x6C 11, 0.00587143, STDIO, stdout, 0x69 0x65 0x6E 0x74 0x20 0x20 0x20 0x20 12, 0.00587204, STDIO, stdout, 0x20 0x0A 0x00 0x00 0x00 0x00 0x00 0x00 13, 0.00588006, STDIO, stdout, 0x0D 0x53 0x65 0x6E 0x64 0x69 0x6E 0x67 14, 0.00588113, STDIO, stdout, 0x20 0x65 0x6D 0x61 0x69 0x6C 0x2E 0x2E 15, 0.00588173, STDIO, stdout, 0x2E 0x0A 0x00 0x00 0x00 0x00 0x00 0x00 16, 0.00593655, Net_SMTP, SendMail, recipients=1 17, 0.00626456, Net_ETH, LinkDownError, no-link 18, 0.21044865, Net_ETH, LinkDownError, no-link 19, 2.21036563, Net_ETH, LinkDownError, no-link 20, 2.91042894, Net_DHCP, ClientState, state=INIT 21, 2.91045321, Net_DHCP, SendDhcpMessage, type=DHCP_DISCOVER bcast=1 22, 2.91066462, Net_DHCP, NextState, state=SELECTING 23, 2.91300110, Net_DHCP, ReceiveFrame, server=10.0.2.17 len=300 24, 2.91304104, Net_DHCP, ClientState, state=SELECTING 25, 2.91308409, Net_DHCP, DhcpOfferReceived, 26, 2.91309079, Net_DHCP, ShowServerId, server_id=10.0.2.17 27, 2.91311357, Net_DHCP, ShowOfferedIpAddress, ip=10.0.2.157 28, 2.91311873, Net_DHCP, SendDhcpMessage, type=DHCP_REQUEST bcast=1 29, 2.91328019, Net_DHCP, NextState, state=REQUESTING 30, 2.91706581, Net_DHCP, ReceiveFrame, server=10.0.2.17 len=300 31, 2.91708329, Net_DHCP, ClientState, state=REQUESTING 32, 2.91709156, Net_DHCP, DhcpAckReceived, 33, 2.91709607, Net_DHCP, ShowAssignedAddress, ip=10.0.2.157 34, 2.91711923, Net_DHCP, ShowLeaseTime, time=171072 35, 2.91715024, Net_DHCP, ShowNetMask, mask=255.255.0.0 36, 2.91716467, Net_DHCP, ShowGatewayAddress, gateway=10.0.2.2 37, 2.91716525, Net_DHCP, ShowDnsServers, pri=8.8.8.8 sec=8.8.4.4 38, 2.91718709, Net_DHCP, NextState, state=BOUND 39, 4.21523585, Net_DNS, RecordTypeNotSupported, rr_type=TYPE_CNAME 40, 4.21535812, Net_SMTP, ConnectIp4, ip=74.125.141.108 port=587 41, 4.89372711, Net_TCP, NoRetriesLeft, sock=1 42, 4.89384738, Net_SMTP, SocketAborted, sock=1 43, 4.89390436, Net_SMTP, ClientCloseSocket, sock=1 44, 4.99372952, Net_SMTP, ClientDone, cb_event=Error 45, 4.99384296, STDIO, stdout, 0x0D 0x53 0x65 0x6E 0x64 0x20 0x65 0x6D 46, 4.99384931, STDIO, stdout, 0x61 0x69 0x6C 0x20 0x66 0x61 0x69 0x6C 47, 4.99385004, STDIO, stdout, 0x65 0x64 0x2E 0x0A 0x00 0x00 0x00 0x00
It looks like you have an invalid system clock configuration. Message 40 starts the TCP connection. If the remote host does not respond, TCP resends the SYN packet. It takes 15 seconds before the message 41 is recorded.
If I calculate the delay, the kernel clock is 25 Mhz, but it is likely to be set at 600 MHz. The errors you mention are the result of invalid clock settings.
Thanks for letting me know about the clock settings. Do you have any advise on my earlier message about the bare metal environment?
My questions are: 1. Will this mbedTLS SMTPS application work on a bare metal environment without a RTOS? 2. Does mbedTLS require an outside API, such as Windows API, to run TLS? 3. Is it possible to move over all the source code to the Arm9 Keilv4 project and get it to work with mbedTLS SMTPS? 4. I read that some TLS certificates require a date verification and my Arm9 embedded system does not keep track of the current date and time, is this involved in SMTPS? 5. Can I just copy over all the source and header files from my working SMTPS Arm Cortex 7 project to my bare metal Arm9 project, including all the RTOS and mbedTLS files?
For a bare metal implementation, mbedTLS requires a socket API. It does not matter on which platform. The network socket interface is in the "net_socket.c".
For MDK middleware, the interface is in:
C:\Keil\ARM\PACK\ARM\mbedTLS\1.6.0\MDK\library\mw_network\net_sockets.c
The SMTPS client demo example is in:
C:\Keil\ARM\PACK\ARM\mbedTLS\1.6.0\programs\ssl\ssl_mail_client.c
You can use those as a reference. However the SMTPS client demo example from mbedTLS pack is only a basic implementation and does not support attachments, multiple recipients, address friendly names and different character encodings.
Note that MDK Middleware version 7 uses a different mbedTLS interface, which allows integration into the MDK MW as an external component.
The latest version of the MDK MW that supports ARM9 devices is Version 4. Subsequent versions only support Cortex-M devices.
And NO, you can not simply move the code to ARM9 device and expect it to work. The SMTP client in MDK MW version 4 is very limited, has a different API, and does not support SMTPS.