Text Encoding and Linux
#1

I'm not sure if this is the correct place for this thread, but I did feel it was necessary for me to pass on a bit of knowledge I came across while working on a recent project of mine.

While trying to utilize PHP scripts as a simple way of creating an API for our MySQL database (to allow all our servers/features to authenticate users), I was kept in awe at one point where all new scripts I created failed to execute even in the slightest! You could get one line to operate but nothing more. This was increasingly frustrating as the console output was very misleading.

When submitting a POST request via SA-MP, I would get HTTP_ERROR_MALFORMED_RESPONSE. Now this in itself was technically correct, however I guarantee you could be staring at this for hours without knowing where the error lies, just as I did.

Amazingly, it was sheer luck that lead me to the discovery of my problem. Linux itself does not seem to have the ability to recognize unicode linefeeds (or Windows linefeeds, i'm not quite sure. I am not an expert in the field of text encoding.)

My issue was that I was saving the scripts with Notepad++ and then transferring them directly to the server. This had worked for me prior to this incident and it will still work intermittently now. However on occasion I would face a recurring problem of failed execution.

I actually had a flashback to when I tried to make a batch editor not 5 years ago, and being horrified as none of the scripts my program created would run. I actually discovered that when editing the file, all the batches were prepended with a " " automatically. As DOS does not support non-breaking space characters, the script would refuse to run.

With this in mind, I opened one of my scripts in standard notepad. To my suprise,

This (notepad++)
PHP код:
<?php
    
echo "testtesttest";
    echo 
"test2test2test";
    echo 
"test3test3test";
?>
Appeared like this!
PHP код:
<?php    echo "testtesttest"; []  [] echo "test2test2test";  [] [] echo "test3test3test";  ?>
I deleted the random boxes, restored the structure of the file, saved it in ASCII format and reuploaded it. It ran
PERFECTLY!

So tl;dr, if you are getting HTTP_ERROR_MALFORMED_RESPONSE and you run a linux webhost, make sure you open, edit and save your PHP script in standard notepad before pulling all your hair out!
Reply
#2

Notepad++ is a powerful text editor. You can see the end-of-line symbols by choosing View -> Show Symbol -> Show End of Line or Show All Characters. Windows format has [CR] and [LF] at the end of lines, Unix only [LF]. You can also change the format of whole text file by choosing Edit -> EOL Conversion -> (format). Just if somebody didn't know yet
Reply
#3

That's incredible...

I just opened up all the old dodgy scripts and displayed their end of lines.

all broken ones only had [CR] instead of [CR][LF]

Thankyou for pointing that out!
Reply
#4

Sorry for bumping.
But on CentOS 6(.2) I had problems executing self made .sh files, I read something about dos2unix and I yum installed dos2unix .
I have tried it and the script was working, maybe this is some kind of same Text Encoding problem?
(dos2unix is in the base repo)
Reply


Forum Jump:


Users browsing this thread: 1 Guest(s)