Tag Archives: php

Request Loop

It’s been a while since I last posted…

There is always a reason for that. My reason was a sum of many different variables. Just as the great mentor said, luck is the sum of many coincidences, that’s what happened in my case as well.

Where do I begin?

Jobwise: Capital controls, working day and night, a lot to do and no time to do it…

Blogwise: I had a very strange setup with my blog (and a very very outdated one I might add). Since I am using Heroku, they decided to change their stack and migrate from Cedar 10 to Cedar 14 . Ok I said what the hell lets do it.

Alas, I had a serious problem with libssl0.98 which was built inside my php module and was not supported in Cedar 14. (whoever wants to do the upgrade have a look here first).

Long story short I fixed it, and I also found that many posts I did with various hacks for the pg4wp plugin were incorporated into a single release from kevinoid : here

I will contribute also into some changes that have to be taken into account since the module is quite old and I have previously stated that it’s not at all well written.

That’s not the main point of this post though.

I wanted to share an experience I keep coming across lately.

Now according to popular trends we are experiencing (and will experience in the future) a huge bloom of the microserviced architecture. This guy here explains how and why they decided to go for the microserviced architecture.

I agree. There are many benefits when having a monolithic single (and obsolete at times) repo for web applications. It is a nice solution when your company is scaling, and you have to maintain a lot of different parts. Especially if you have different teams and each team wants to “do their own thing” about a solution.

However it’s not the solution to Everything!

I will elaborate more:

I recently had to debug an http step based procedure (client requests this page, books this ticket, goes there, etc.) that was using 3 different instances of different technologies over http. The one was python and wsgi, the second was php with apache and the third one was ruby with unicorn.

Try to debug this. I dare you. Seriously. I had in my local setup all 3 different instances running with 3 different IDE’s and all running their debugger. Ok, ok you say that Docker will simplify the installation. I agree it does, but it does not help the debugging at all.

The most important thing though isn’t the debug/testing of many different apps over http.

It’s the HTTP by itself.

And believe me, I have seen a lot of “Senior” Devs falling into the same trap of API’zation and doing over and over the same architectural error.

The Request Loop

You won't guess how many time's I've seen this happening...
You won’t guess how many time’s I’ve seen this happening…

Consider the following diagram:

This is the actual loop - when one request is still open, another comes along, and things get messy...
This is the actual loop – when one request is still open, another comes along, and things get messy…

The Browser  sends a request to the Frontend app. Now the Frontend App could forward it (or change it a bit) to the Backend App.

In our setup the backend app was a PHP app.

Now since PHP by default does not support threading (not pthreads), each HTTP request is a different PHP thread, served via apache.

This is very complicating, since you keep a connection (process) open and you open another one which could (at some point maybe) rely on data from the first one. You cannot access that data in between processes.

Not to mention that, you can not either debug this thing, since you insert a break point in the first request procedure, and the second request (which happens a few ms after) is being served without the debugging stopping at that point.

My point is that when you decide to go Microservice’d

Try to avoid request looping, when you need to do something that is synchronous. Or, use something different. Do threading. Use a message queue, or something else.

You will be surprised how much time you will spend trying to debug and understand what is wrong in this set-up.

I will close with the following meme:

Some people, when confronted with a problem, think, “I know, I’ll use threads,” and then two they hav erpoblesms.

WordPress SEO Sitemap and Heroku

Maybe you have read at a previous post about Heroku and WordPress

that we have been using PostgreSQL as a persistent storage for this current blog.

Aside from the problems with our hosting as it is, from the fact that we could not (and still cannot) run add-on PG4WP effectively with Heroku and WordPress, we found another rather serious problem.

Last night I was browsing Google’s Webmaster Tools

and I found out that my sitemap was not working properly.

Despite the fact that the link was loading (http://www.must-feed.com/sitemap_index.xml) if you got in and tried to load the posts sitemap (http://www.must-feed.com/post-sitemap.xml) it responded with a Not Found (404) page.

Post Sitemaps was Not Found (404)

Looking around the Heroku logs (in terminal write heroku logs) and I found out this really interesting error :


[error] WordPress database error ERROR: date/time field value out of range:
"0000-00-00 00:00:00"\nLINE 1: ...ssword = 'xxasdf' AND post_author
!= 0 AND post_date != '0000-00-0...\n
^ for query 
   SELECT COUNT(ID) FROM wp_posts
   WHERE post_status IN ('publish','inherit') AND
   post_password = '' AND post_author != 0 
   AND post_date != "0000-00-00 00:00:00" AND post_type = 'post' 
made by require('wp-blog-header.php'), wp, WP->main, 
WP->query_posts, WP_Query->query, WP_Query->et_posts, 
do_action_ref_array, call_user_func_array, WPSEO_Sitemaps->redirect, 
WPSEO_Sitemaps->build_sitemap, WPSEO_Sitemaps->build_post_type_map

I digged the code a bit and found out

that the problem was being caused by a query which was ran from WordPress SEO Yoast Plugin.

This query :


SELECT COUNT(ID) FROM $wpdb->posts {$join_filter} 
WHERE post_status IN ('publish','inherit') AND post_password = '' 
AND post_author != 0 AND post_date != "0000-00-00 00:00:00" AND post_type = %s

Had an invalid date for PostgreSQL database.

Not to mention that also, the guy who wrote PG4WP

the module that connects Postgres with WordPress (to be honest that is heavily resource consuming) had thought of sanitizing the query from these cases only at INSERTS:

$sql = str_replace( "0000-00-00 00:00:00", "'now() AT TIME ZONE 'gmt'", $sql);

EDIT: I found out that the above line while writing this post
was SQL Injecting the code which was escaped by  the back slashes of gmt and caused again Postgres to fail so I changed it properly. The ajax action of post creation was calling the wp_insert or wp_update method which in turn was trying to insert a new post. I will come back with another update since I cannot escape single quotes somehow…

And not on SELECTS and UPDATES.

So all I had to do was add a new line at 290

of file : /wp-content/pg4wp/driver_pgsql.php


$sql = str_replace("0000-00-00 00:00:00", "1977-01-01", $sql);

to sanitize the SELECT and the posts sitemap page (and xml) was being loaded properly:

EDIT: I have also found out that the same bug applies with the press_this.php functionality and you cannot use it (you need to sanitize this as well).

EDIT 2: There are numerous errors during rewrite from different modules. I will come up with my patches at a new github repo. Recently I found out that Jetpack also had a problem, more on that later.

Not Found error disappeared and the Posts entries were generated in the xml.
 Finally after all those I have to say

that the way WordPress uses the Database is the least RIDICULOUS.

Having worked with many systems on web, I suggest to move to a more database agnostic framework, such as pdo_mysql.

Parse Error PAAMAYIM NEKUDOTAYIM

Now I get it. PHP is a collective language created by multicultural people all over the world.

 

Well, ok. Which makes you think that all errors should be properly explained in a common language. Since we do not all speak the same language (apart from math that is) English is a good candidate.

Surely it is not Greek. Surely it is not Persian. And by any means its not Hebrew either.

Yet, this :

PAAMAYIM NEKUDOTAYIM

Parse error: syntax error, unexpected T_PAAMAYIM_NEKUDOTAYIM

Okay. If you google it , turns out that PAAMAYIM NEKUDOTAYIM means double colon (::) in Hebrew.

Turns out that the two main guys who wrote the Zend Engine that translates PHP to executable code, were from Israel.

And after all those years this has somehow not changed. Which eventually will make your eyes bleed. Since it would be the same as writing: “Dipli Anw Kai Katw Teleia”