Uploading data to the server [wbwit II]

Uploading the data gathered from the web-based version of the word identification task (i.e., lexical decision – see this post) to a MySQL database was more challenging than expected. The problem is that there is a limit on the size of data which could be transfered with an ajax XMLHttpRequest (ajax call from now on) from the web-page to the host server. It took quite a bit of trial-and-error to discover that the problem was a limit to the size of the ajax call in the server of our host provider (I estimated the limit to be around 1 kB). In fact, when testing the ajax call on localhost it was all time successful, but not on the hosted server. Since one might wonder about the size of the object I was trying to send, I will quickly describe it. The data is a javascript array of 100 objects. Each object contains the word to be displayed (name attribute), the word’s index in the words database (wordIdx attribute), whether the word is a word or not (isword attribute), the reaction time (RT attribute, added after the response), the key which was pressed (keyPress attribute, added after the response). It looks like this:

var words = [

  {name:"abstair", wordIdx: 1, isword:0, RT:456, keyPress:119},


  {name:"abstract", wordIdx: 99, isword:1, RT:654, keyPress:109}


I could estimate the size of 100 such objects to be 6600 bytes, computed following this ) stackoverflow answer.

To overcome the server limit I thought of looping through the array of objects and uploading to the server one object at the time. So the ajax call1 became:

function populateDatabase(words)
   nRequests = words.length;
   url = 'data2db.php';
   for (i = 0; i < nRequests; i++)
      var xmlhttp = new XMLHttpRequest();
      xmlhttp.open("POST", url + '?prikey=' +
         JSON.stringify(words[i]), true);

However, this solution is problematic when the results of different participants are uploaded to the server simultaneously. In fact, responses of different participants could be mixed up into the database. But PHP comes to rescue, and by setting a sessionID I could make sure that each participants data is stored with their unique ID. Additionally, since the session ID is 26 characters long and it would be a waste of database space to repeat it 100 times I decided to store in a separate database. Then the php code in the ajax call retrieves the numeric index of the respondent and uses this numeric index to store the reaction times and accuracy of each word displayed. Moreover, the php session ID will also allow to identify the respondent responses to the questionnaire which I am planning to administer before and after the word recognition test. The php code in the file data2db.php is below:

$obj = json_decode($_REQUEST['prikey']);

$servername = "localhost";
$username = "rootino";
$password = "not2day";
$db_name = "treasureChest";
$conn = new mysqli($servername, $username, $password, $db_name);
if (mysqli_connect_errno()) {
    printf("Connect failed: %s\n", mysqli_connect_error());
echo "Connected successfully";
echo "<br>";

$sql = "INSERT IGNORE INTO `respondent` (`uniqueID`) VALUES 
if ($conn->query($sql) === TRUE) {
	$sql = "SELECT id FROM respondent WHERE uniqueID = 
	$result = $conn->query($sql);
	if ($result->num_rows == 0) {
		echo "0 results";
	} else {
		$row = $result->fetch_assoc();
		$last_id = $row["id"];
} else { echo "Error: " . $sql . "<br>" . $conn->error; }
echo "got last id";

$sql = "INSERT IGNORE INTO `{$db_name}`.`responses` 
  (respondentId, wordIdx, RT, keyPress) VALUES  
if ($conn->query($sql) === TRUE) {
	echo "Record added: " . "<br>";
} else { echo "Error: " . $sql . "<br>" . $conn->error; }
echo "added records";

The complete code is available on this github repository.

1. The GitHub repository contains an updated version of the ajax call and two php file. This is only a performance change so that the uniqueIndex of the participant is retrieved only once instead of 100 times like in the case above.

Uploading data to the server [wbwit II]

One thought on “Uploading data to the server [wbwit II]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s