What? Got sum'tin against Pink?

How to make a Web Service In FreePascal?

Last Updated on Wednesday, 25 November 2009 16:03 Written by Administrator Wednesday, 18 November 2009 04:27

I was asked on the FreePascal forums how I went about making my web server. This is a loaded topic for sure, but I wanted to share my post with folks who might be interested in the subject matter. This was written in the context of specifically answering the posters question. Enjoy...

 One bloke's Approach...

...to writing a Web Server or Service in FreePascal

Dimitrij Kowalsky Asked: "Jason: Do you want to tell me that you have implemented that service from scrach without using Apache to hosting it?

You had to put a tremendous amount of work into it....

Could you describe process of creation such service in some more details?

I am thinking about creating a tutorial for other newcomers (like me) so it could help promote FPC and bring more users into community. Of course I need learn it myself firstly ;-)"

ut oh.. this Jason fellow wrote a preface, break out the tea... B^)

Preface

Hi Dimitrij,
I would love to share how I went about it, and what I'm still doing; you are absolutely correct in that I have put a lot of effort into it.

To start, I have been a fan of FreePascal for a decade or so now. When I got started with FreePascal, the authors were more accessible than they are now; I imagine their lives are just busier now like mine is with work, family and wanting to accomplish a million things with time always being the enemy.

I think it's important to mention this because as a new user of FreePascal I was only scratching the surface until Marco, Daniel, Jonas and even Florian (Michael and Jonas especially) were very influential and helpful to me then; they remain influential and still help me now and again. My point is that these guys took an already excited gung-ho programmer and didn't crush my spirit but instead pointed me in the right direction and increased my enthusiasm enough where every FreePascal limitation or obstacle was just another challenge; a file or library to locate or some code to write. I appreciate this opportunity to answer your question with vigor in the hope that it helps you in much the same way those guys helped me!

Getting Started with Object Oriented Programming

First off, all I was originally trying to do was learn classes and object oriented coding. I understood it fairly well until I ran into a snag writing some double linked list code. I asked about it in mailing list and Michael not only answered my question but wrote about four different examples demonstrating specific points about object handling and pointers that were specifically written to show me what I was doing right, wrong and where I could go from there. That was pivotal! So, let me stress that to this day I'm still using those same code bases and that knowing how to make classes from scratch is extremely important for when you want to use those included in the free component libraries etc. Point: Understanding classes from the bottom up is pivotal if you want to design something that scales well; this doesn't mean you need to absract everything to the Nth degree, but it does mean you should worry about memory leaks, and proper object instantiation and destruction with every line of code you write so issues don't get away from you.

CGI - Common Gateway Interface

Next I was trying to write CGI applications, which I'm still convinced is the best interface out there providing your CGI "program" is a thin client that communicates with your big application and the CGI part is really just to get data from the webserver and back to it. To much more and you suffer from what ails CGI programs generally: They use up resources making a shell, loading a program and firing it, then making database connections etc just to render a web page. My approach has been to have an application (Big Boy) running waiting for work; the CGI application (little boy) gets fired up from the web server and tells the Big Boy what needs to be done (inter process communication - using TCP IP these days for portability) and then the results are sent from the Big boy to the little boy who sends the result back to the web server. This is much like the approach FastCGI uses without needing to compile code or modules directly into the web server leveraging a standard (CGI) that works for almost every webserver in existance. All this might be overkill for your current task, but I'm hoping the read is interesting nonetheless.

To get CGI working without any prior CGI experience was quite a task at first, but once I started to understand it; it naturally started to get easier. I actually got started with the CGI samples that come with FreePascal because the RFC documents just didn't help me in the least. Learning the CGI interface from code and from this web site: http://hoohoo.ncsa.illinois.edu/cgi/overview.html helped me tremendously.

For your project where you want a service that is secure, gets a request and returns a request over the internet, I think CGI is probably a perfect fit for you.

• You do NOT have to deal with Synapse or other networking libraries
• You do NOT have to worry about SSL
• You do NOT have to encrypt anything unless you have special reasons
• You do NOT have to write a huge application with lots of infrastructure

CGI programming is akin to writing a console application: stdin and stdout. The drawbacks of CGI are startup is expensive. Frankly, on today's machines, if your application isn't to fancy or is spread accross multiple CGI programs that individually are small but make up your system; you will probably do wonderful. Before you see my words and become trepid; please try making a "hello world" CGI application, serve it on your webserver and benchmark it with apache benchmark. When you see how fast it performs and consider likely realtime usage; I think you'll be impressed. Frankly it's faster than PHP; however PHP has one advantage that most web installations already have it loaded in memory and the webserver hands the request directly to PHP so there are no clock cycles lost to spawning a "shell" to launch you executable. However.. once your CGI is loaded; the compiled Freepascal code (providing you write reasonably efficient code) will out perform PHP script like a race horse out performs a donkey at the track.

If your CGI application does grow to a size where load time is really a problem, all is not lost. Your core application code can be refitted to a standalone homebrew webserver, or can be retro fitted to a design like I mentioned above with the BigBoy-LittleBoy paradigm or you can make your code work with FastCGI (something I found to be more trouble than it was worth: portability, ease of use, etc. though I like the concept... shoot - I had the BigBoy-LittleBoy thing working before I even knew about FastCGI. It wasn't rocket science either: slow load time? Minimize load time... See? Easy logic. Execution was a bit trickier, but once I nailed down a few different ways to do InterProcess Communication (Drop files using block write/reads, FreePascal IPC unit, pipes and settling on the network stack) things got pretty slamming! I wrote a big realestate application called "Wimble" (as in auger - dig though the data) it was the fastest thing going for shopping online for houses in Connecticut USA, faster than the MLS or Multiple Listing SService site of the Realtor® company who was actually providing the data to me! In fact, only 6 months after rollout they changed all the rules to getting their data and made it really hard for folks like me (actually was working with a realestate buddy of mine at the time) to do what we did. I like to think that we were just beating the living snot out of their web site: fast as heck, easy, no advertising, just pictures of houses and land, quick searching tools, and we had users in droves. Ah well... One win for Freepascal for sure!

The SSL you do not really have to worry about with CGI because if the web server is set up to use SSL, and your CGI application is available through the SSL, you completely inherit all the SSL stuff without having a line of code even needing the letters S.S.L. in it!

In CGI, you read stdin to see what has been asked of you, parse out the URL sent if you like, then you write stdout your web page or service response.

How about a web server?

Well, to make your own webserver here you have networking and, in your case, SSL issues will need to be dealt with. Eventually you may even wish to delve into multi-threaded programming.

Note on Multi-threading: You do not necessarily need to tackle the multi-threading issues when you are just getting started making your first server, however it would might be wise to understand how multi-threading works a little so you can at least attempt to write code that will port easier to a multi-threaded environment at a later date; specifically variables need to be set up very carefully. With a multi-threaded application, you can no longer just have global variables and expect reading and writing to occur without unexplainable crashes that are very difficult to track down due to their seemingly random nature.

Ok back to the web server stuff!

When I got started I knew I needed to cover more things but I was admittedly shocked at how many layers there were to this onion: TCPIP, Ports, Swapping ip address bytes around (Say 127.0.0.1 is the IP, you have to swap it around in the structure holding the bytes or octets so its like 0.1.127.0 for it to work right?!?!?! What the heck? LOL). Then you have the whole HTTP protocol sitting in there next; as there atr certain rules you must follow (and many optional rules) to get the most basic HTTP stuff working. On top of this, there are MIME types to consider, potentially charset issues (UTF-8 is great! LOL) which goes alongside internationalization, generating proper Time and Date strings.. with GMT times.. It's really quite daunting and I didn't make ANYTHING worth mention until I 100% disected and studied, changed, broke, and fixed this FPC webserver application I found from a real nice guy in Russia. This guy was actually quite pleased with my efforts and when I was finally able to tell him how I used his program as a guide and made my own multi-threaded web server. He told me his program was an experiment and he was happy someone was able to get so much from it. Yeah, you probably want to check this thing out for yourself. Hang on.... Ok, it's called nYume and I placed it on my FTP server so you could find it easy. Note there is a zip in there called tcpz.zip and that is not related but came from the contributed units page here on the FreePacal site in the internet area (which has a lot of stuff to check out). Anyways, here is the link to nYume: ftp://jegas.net/archives/coderelated/freepascal/nYume_WebServer/.

that little bit should prove handy... I'm looking for some of my old CGI stuff.. honestly, there is some VERY old code I wrote in the contributed units area of this website that I've been unable to get at and maintain since I lost that account. There is however some stuff I shared, may be same version maybe a bit newer also on my FTP stp here: ftp://jegas.net/archives/coderelated/freepascal/25885_SageAPI_PHeaven_V1.zip It's called the odd filename because the file is really SageAPI v1, but I submitted that to Programmers Heaven's years back and the file ID was 25885... but I digress. There is some CGI code in there I believe... shoot the early reditions of my double linked list classes etc are in there also I believe.

Well, I've ranted quite a bit.. I hope to hear back from ya and I hope you enjoyed this post. I have more to tell but I figure this is more than enough for the moment. Take care! :)

--Jason P Sage

My Sites: Personal and Work.