Hi
I'm having some problems reading from the serial port. ASCII > 127 are displayed as ' '. I know this is an encoding issue, and I've read other post here on the same issue, but I haven't found a solution. I tried using _serialPort.Encoding = Encoding.GetEncoding(28591) which didn't make a difference. I've also tried to set the Encoding property to Encoding.Default and loads of other encodings with no luck. For some encodings, characters over 128 are replaced by a square, sometimes a newline or some other weird character. Based on the documentation for the GSM modem I'm communicating with, i should use USC2 alphabet, and Unicode is the closest match. However, when using _serialPort.Encoding = Encoding.Unicode, there is nothing to read from the serial port. I can't find any reason for this, and been struggeling with this problem for a few days.
I'm polling the modem for resonse. This is the code I've been using:
_serialPort = new SerialPort(PortName, BaudRate, Parity.None, 8, StopBits.One);
_serialPort.Encoding = Encoding.Unicode;
_serialPort.NewLine = "\u00C1"; // or "/r/n"
_serialPort.ReadTimeout = 300;
_serialPort.WriteTimeout = 50;
_serialPort.DtrEnable = true;
_serialPort.RtsEnable = true;
...
...
while (String.IsNullOrEmpty(replyString) && retryCount++ < attempts)
{
// Only sleep if no data is read
// Sleep at the beginning of the loop to let the modem have time to return data
if (String.IsNullOrEmpty(replyString))
Thread.Sleep(sleepTime);
//replyString = CleanString(_serialPort.ReadExisting());
Byte[] bytes = new Byte[_serialPort.BytesToRead];
_serialPort.Read(bytes, 0, _serialPort.BytesToRead);
replyString = CleanString(_serialPort.Encoding.GetString(bytes)); // return string with newlines etc removed
}
Why is it that there is nothing for me to read from the port when using Unicode, when reading is successful (except invalid characters) for other encodings
Thanks in advance
Bjornar
Serial Port - Encoding problem
Vladimir Fedorov
I haven't found a solution to the problem when setting the Encoding property of the SerialPort class to Unicode. The workaround has been to set it to ASCII, and handle the encoding of bytes myself. I have found this to be the best solution as the application I'm developing will be sending and recieving SMS messages, and using Unicode, text messages will only be 70 characters long.
New problem - part 1
The current problem I seem to be having is when messages are being sent using unicode. One of the devices I'm sending SMS from, is a HP iPAQ with Windows Pocket PC 2003. When I send an SMS containing the following characters: { } [ ] ^ ~ | € to the application, the PDA sends these as Unicode (http://www.dreamfabric.com/sms/default_alphabet.html). They are not sent as GSM character set. Checking Inbox->SMS->Accounts->Accounts->*SMS, there is a checkbox with "Send SMS messages using Unicode when necessary". With this box checked, I send a message containing the character "[" to the application, and it is recieved as byte[] 48, 48, 53 which is 005 as text. I've tried using Unicode (little and big endian) to transform them to a different encoding, but that only result in junk. When unchecking "Send SMS message using...", the message is recieved with some unknown characters. I haven't looked too much into this situation, so I just want to know if anyone know why the unicode message is sent as it is. When sending a message containing these characters from a Nokia 6820, the message is recieved just fine with escaped characters (and I can convert them to normal text)
New problem - part 2
When sending a message containing the escaped (27, xx or 0x1B + character) characters listed in the default alphabet (see link in part 1), they are converted to the corresponding escaped sequence, so { is converted to 27, 40. The problem with this, is that the SerialPort treats byte 27/1B as <ESC>, and the sending of the SMS is aborted. I usually send 0xA1 to the modem to send Ctrl-Z to make the modem send an SMS.
Is there a way I can make the serial port (or perhaps I have to change something in the modem) treat 27/0x1B as a normal character when I want it to, and as the actual escape key when I don't
From the docs for the modem to send SMS:
AT+CMGS=+4712345678<CR>
text is entered after > is recieved
<ctrl-Z/ESC>
"Sending can be aborted by entering <ESC>. Of course, the message will not be sent, though the operation is acknowledged with OK." - this is the case for New problem - part 2
Thanks in advance
Best regards
Bjornar
Thogek
If you don't see the zeros, IBM437 might do the trick. If the modem sends any binary data, you ought to see it untranslated and need to use the byte values without trying to convert them to a string. Otherwise, the modem might use UTF8.
If you do see zeros, pay attention to their location. If you see a zero first, the modem is sending big-endian Unicode characters. Your PC uses little-endian, set the encoding to BigEndianUnicode.
The charmap utility can help you diagnose what glyph is displayed on the screen for a character code. Run it with Start + Run, charmap. Note that fonts on Western PCs have but few glyphs for the entire Unicode range. If you try to display a code which the font does not support, you'll get a square. If you get a character from the modem that the encoding does not support, you'll get a question mark.
Please let us know how you make out, this has been a popular topic on the forums without any good answers.
Kevin Williams
abh2005
Sorry for the late reply. The raw bytes read from the modem started with 00, but was unable to fix the problem.
I found an AT command for the modem to change the encoding of the data to USC2, however that didn't seem to make any difference. I had to find a different solution to the problem. SMS messages use the GSM 03.38 encoding, which is mostly similar to the ASCII encoding, but with slight differences ( http://www.dreamfabric.com/sms/default_alphabet.html). I ended up writing an Encoding for this character set, which seemed to be the simplest solution. It seems to work, but I'm having some problems when reading text from the modem containing escaped characters where the message string contains only numbers and a few letters. I am currently working on Encoding/Decoding the escaped characters. I can't assing the new encoding to the serial port, due to restrictions on the serial port Encode property, so I had to use a manual approach. Will continue tomorrow.
Thank you for your feedback. Will try to get back to you on the final solution.
Best regards
Bjornar