A strange learning path bug… fixed

See update below for the fix

I’m trying to get my head around a super-tricky bug since last Friday. Basically, the bug happens only on one portal and with a specific learning path imported by Oogie.

The bug appears due to a loss of session before the generation of the table of contents.

Something like this appears:

Fatal error: Call to a member function get_theme() on a non-object in /var/www/yourportal/main/newscorm/lp_toc.php on line 39

The bug is not in the Oogie conversion (no strange characters) as the same content works on another portal on the same server.

The bug is not a PHP configuration problem either, as again, another portal on the same server, with the same PHP configuration, works perfectly.

The problem seems to reside in the time the newscorm tool needs to generate some of the frames. Apparently, one of the frames loads before the others, and looses the session data, but only on this portal.

I’m not out of things to try out, but this one is really a painful one.

2008-07-04 Update

After a long week of interrupted searches, I realised (mostly thanks to a conversation with Arnaud) that the bug was caused by an overflow in the session size capacity.

Now most of you PHP developers will think “mmmmh, is there actually a way to limit sessions file size on disk?”. Well, the answer is “no”, not in the predefined PHP5 options, anyway. However, there is kind of a special condition here: the portal on which this error occured had the setting configured to save the sessions inside the database, using the dokeos_main.php_session table.

The thing is, inside this table, we defined the field to store the session as a “text”-type field (or blog, which is the same for MySQL 5). However, I did an enormous mistake in thinking that a “text”-type field could actually store up to 4MB. It can’t. It just stores a maximum of 16KB.

This size should be enough, in most situations… yes, but not in the learning path object situation. Learning paths are very large objects with a structure set to limit the number of queries made on the database. Even then, however, it coped nicely with *most* learning paths, until this 56-pages learning path appeared. Apparently, the structure was too large and went over the 16KB limit, making it unproper to serialization into the database field.

So basically, turning the field to a MEDIUMTEXT field fixed the problem:

ALTER TABLE php_session CHANGE session_value session_value MEDIUMTEXT NOT NULL;

To make things clear, the problem appeared to be a loss of session, somewhere between the script where the learning path object is built and the first script using it straight from the session.

Debugging the second script showed that the session was recovered (or rebuilt), but without that large element, which just couldn’t be stored in the database. This made it really difficult to track down, as nothing is telling you anywhere (even with full debugging on) that the problem comes from the session size and the database field limit…

Of course, this has all been fixed in the Dokeos code for the next version of Dokeos, so you don’t have to worry too much about it, it will just get updated automatically.

Installing the videoconference - missing data in documentation

We just realised that through an unfortunate series of events, I had removed a line that Arnaud had added in the documentation on the videoconference installation process. It was missing the part where you download the dokeos-conference.war file and copy it to your webapps directory. The audiorecording is there, but not the videoconference.

In any case, the documentation on our website is correct (except one occurence of the name of the file but that’s obvious) and we have updated our code sources:

http://www.dokeos.com/doc/installation_guide.html

Sorry about that…

Improved SCORM content migration from 1.6.5 to 1.8.6

I’ve had to review the SCORM content migration from 1.6 to 1.8 just today, and really, this is not something I’m enjoying. The complexity of migrating the old structure is enormous due to the lack of structure we had in 1.6, that lead many to launch a SCORM content through a simple link, rather than importing the SCORM content through the SCORM tool (yes, 1.6 allowed that and no, we are not going to allow this again because it makes a mess of the internal structure).

So basically, what have I done?

First, I made sure the links from the course introductions *and* the course links were both treated as possible sources of learning paths. There was something checking those links before, but they were only treated if there was a corresponding correctly-added learning path in the tool.

Second, I realised that, when checking if one of these links really pointed to a SCORM file, I didn’t actually take into account the fact that the old repository was placed, for the time of the migration, into another directory. So I was checking into the receiving courses directory (which at this point of the migration process is still empty) for a file that was in the old courses directory. This, of course, prevented the migration of such SCORM contents to be succesful. This has now been fixed.

These change comes a bit later than the official 1.8.5 stable release, so you won’t benefit from them if you download a stable version from the website now. However, you can still download the file and apply it on the main/install/ directory of the 1.8.5 package *before* you upgrade from 1.6.5 to 1.8.5.

Et voilà, I hope this helps the million of people out there still having held on before migrating from 1.6 to 1.8.

While doing all this, I also realised that some of the courses tools were not correctly moved from the Production section to the Interaction section, but I didn’t find how to fix it. The SQL query is there, it is apparently not working. This is just a matter of executing a little script afterwards though, or updating the database (I’ve seen the problem for the forum and the announcements tools).

You can fix it (in every course database) by issuing:

mysql> update tool set category=’interaction’ where name = ‘announcement’;

You can check out the script itself by clicking the “download” link in our subversion repository

Dokeos single database - don’t let yourself get tricked

I wanted to write this post for two reasons:

1. to remind everybody that we (the Dokeos company) continue to vaguely support this mode because we understand it is still more practical when installing it on a free/cheap hosting server
2. to help you avoid the tricky issue in the installation process when deciding which main db name to use

As the first point is pretty much complete, I’ll go straight to the second point. In the Dokeos installation process, the database configuration screen asks you for a prefix and a database name. It’s all OK, but the tricky thing is that the default value that are given are “dokeos_” for the prefix and “dokeos_main” for the main database.

This might lead you to think that the database created will actually be called “dokeos_main”, while in this context of leaving the options by default, it will be called “dokeos_dokeos_main”. Yes, the prefix is added in front, as any prefix is.

I know, it’s not very intuitive so I’ll try improving that for the next version, but at least you’ve been warned!

First annoying bug in 1.8.5: learning path theme feature

Yes, we have our first annoying bug in Dokeos 1.8.5. It has a very restricted scope, and only happens once you voluntarily activate a new function, but it is there…

When you *upgrade* your portal from anything to a 1.8.5, there is a missing bit of SQL in the upgrade process that mean you will not have a column called “theme” inside your courses’ lp tables.

First, let me tell you that you can fix it pretty easily before upgrading, by adding this line to the end of main/install/migrate-db-1.8.4-1.8.5.pre.sql:

ALTER TABLE lp ADD theme varchar(255) not null default ”;

And the conditions to reproduce the bug are these:

1. upgrade to 1.8.5 (the bug doesn’t appear on fresh installs)
2. in a course that existed before the upgrade, go to the course settings
3. activate the option to set a CSS theme by learning path
4. go to the learning paths page
5. edit a learning path’s properties by clicking the pencil icon
6. change the theme used for this learning path
7. click ok

That will display a horrible SQL error (that is, if you left your PHP settings on display_errors=on) that will tell you that the field “theme” does not exist in that table.

You can fix it by manually updating the database (using the field definition above) or writing a script that will update all courses databases to add the field. The writing of this script is left to the reader as an exercise ;-)

Dokeos, new course, error about group permissions

There is a common problem appearing when installing a new Dokeos portal that I have seen a lot recently, so I thought I’d share the details here.

The problem

When installing Dokeos on a cPanel-kind-of-hosting, it might happen that you complete the installation, but when you want to enter a newly-created course, an ugly error appears. Something of the likes of:

Internal Server Error

or, if you are lucky

/…/courses/COURSECODE/index.php cannot be displayed because it is writable by the group.

Either way, those two error message precisely when the URL of your browser shows http://your-domain-name/courses/YOURCOURSECODE/index.php mean that you have a “secured” version of Linux.

This implies that for any PHP script that you want to execute, this message will appear if the script is writable by any other user than the owner of the file.

In a cPanel system, you will see that this file (and possibly the directory it’s in) have write permissions for the group.

The permissions syntax

I’m reviewing basic stuff here, so if you know about UNIX permissions, just skip to the next section.

This is represented by the permissions indicator

-rwxrwxr–

which can be translated as: the owner can read, write and execute this script, the owner *group* can read, write and execute this script, and all other users can read the script, but not write or execute it.

The write permission gives you the right to edit the file, but you need a write permission on the containing *folder* to actually create or remove this file.

Anyway, so the risk is that, by letting too many people access this script, it could be used (and modified) by a cracker (an evil hacker) to execute his code instead of yours on your server.

Fixing the problem now on the server

Now the quick fix is to read carefully the error message and change the permissions accordingly. The server tells us that this script cannot be opened because it is writable by the group. So all we need to do is remove the write permission on this file. Just click on whatever option allows you to change the permissions on that cPanel and remove the write permission for the group.

This should result in your file’s new permissions to look like this:

-rwxr–r–

If the server bothers you with write permissions about the directory, you need to set the directory’s permissions to

- rwxr-xr-x

Execution permissions are needed to read inside this directory (and get to index.php).

This should solve your problem for this course. Now you want to avoid doing that for all courses to come, don’t you? Read on…

Fixing the problem for the future from inside Dokeos

Since Dokeos 1.8.4 (or was it a little before?), we added some settings inside the database that lets you mention what type of permissions you want new files and directories to use. Well, this is precisely one case for which we did that.

You want to head towards your “Portal Administration” tab, “Platform” section, “Dokeos configuration settings” link, then “Security”, then the “Permissions for new directories” and “Permissions for new files” settings. By default, these are set to 0777 and 0666. Considering you want the groups not to have write permissions, and considering 0777 is representing rwxrwxrwx and 0666 is representing rw-rw-rw-, you just want to change these to 0555 and 0444 respectively.

That’s it, you can now create a new course without having to worry about files permissions!

Video activity report for Dokeos

There’s some new stuff on the net in terms of code analysis. If you already knew about the CIA project and the Ohloh.net website, and you liked it, you might get a good interest in code_swarm, a java tool that uses SVN logs to generate a (good-looking) video of the project activity in terms of code development.

I played with it a little bit for you, and this is what it looks like…

The video itself doesn’t look too great once it has been processed by YouTube, but you can get a better quality one from here (about 7MB): http://www.dokeos.com/download/dokeos-activity2.avi

Howto convert AVI to DIVX with mencoder

As I was looking into reducing the size of a video, I felt upon this wiki page, which proved super-useful:

HOWTO Convert video files

In short, you can use something like this for a quick fix:

mencoder <filename.avi> -ovc xvid -oac mp3lame -xvidencopts bitrate=687 -o <output.avi>

or maybe something like this:

mencoder <filename.avi> -ovc xvid -oac mp3lame -xvidencopts bitrate=-700000 -o <output.avi>

(setting a negative bitrate lets mencoder calculate the bitrate automatically to reach a file size of the given negative value in KiloBytes, although this doesn’t work for me…)

No security flaw so far…

It might seem kind of weird to mention it, but I had somewhat of a big scare when Secunia sent us an e-mail entitled “Security Patches in Dokeos 1.8.5″. If my fears had been right, it would have meant that we would have needed to re-package 1.8.5 only two days after the official release. Not something I would have liked to do…

However, it was just a basic “request for information” e-mail asking what security improvements had been added to Dokeos 1.8.5 as we mentioned so in our changelog.

The answer is that we integrated Dokeos 1.8.4 SP1, SP2 and SP3, as well as improved input filtering all over (more than anything in the survey and forum tools).

So we still have a robust 1.8.5, which makes me very happy.

Dokeos 1.8.5 API documentation available

I’ve just finished generating the new API documentation for Dokeos 1.8.5. It’s meant for PHP developers only.

http://www.dokeos.com/doc/dokeos_api_1.8.5/