Web-interface for Delphi Method III

Delphi method, round 2

This post is the third in the series describing the web-interface I created to administer a questionnaire with the Delphi method. In this post I describe the code used to give feedback to the participants in round 2 of the Delphi study on functional magnetic resonance imaging on tinnitus. If you are interested in the previous posts, this post describes the interface for round 1 whereas this post describes (the first part of) the interface for round 2.

The framework guiding the building of the interface was to create reusable code. Because I wanted to display the feedback from the previous round through bar plots, I had to make two graphs for each of the 40 questions of round 1. I wanted to be able to build both graphs using the same function. Moreover, to keep things simple and straightforward, I wanted to avoid to have to process the input arguments given to the function to create the two different graphs. Therefore the plotting function should have been called without passing arguments while keeping the graphs’ content split on the main page. This implied that retrieving 1) group responses and 2) the current user’s responses had to be done with the same XMLHttpRequest.

Let’s look at the XMLHttpRequest first. The XMLHttpRequest sends two identifiers, one for the experts’ group (expType) and one for the current question (qNum). The identifiers are passed through a JSON object to the fetchResults.php page. They then enter the MySQL SELECT statement addressing the database to find a count the responses. The XMLHttpRequest returns a JSON array with the responses of a given group to the current question. The array is passed to the barPlot() function through the datum field of d3.select().

if ( < 41){
  document.getElementById('firstPlot').innerHTML = 'MRI experts';
  document.getElementById('secondPlot').innerHTML = 
    'EEG, MEG, psychoacoustic experts';
  var xmlhttp = new XMLHttpRequest();
  xmlhttp.onreadystatechange = function() {
    if (this.readyState == 4 && this.status == 200) {
      var myArr = JSON.parse(this.responseText);
      d3.select("#second").datum(myArr).call(barPlot()); }};
  var selResps = { 
    qNum:"", expType: 1};
  xmlhttp.open("GET", "fetchResults.php?indexQ=" + 
    JSON.stringify(selResps), true);

The responses from MySQL SELECT are stored into an array which is extended with the response of the current individual if s/he belongs to the current expert group (i.e. if the second select statement does not return empty). The current user’s responses are selected through $_SESSION[‘ppid’]. $_SESSION[‘ppid’] is stored in the PHP session, so there is no need to pass it through the XMLHttpRequest. Since the counter for the question is also stored in the PHP $_SESSION I could have retrieved the current question number from the $_SESSION variable instead of passing it through the XMLHttpRequest. At the time of building the page I did not think about it, but now I realize that trick would have made things smoother… The array returned by the XMLHttpRequest contains the data to specify the height of the bars in the bar plot.

$obj = json_decode($_REQUEST['indexQ']);
$expertType = $obj->expType;
$obj = 'q' . $obj->qNum ;
require '../includeDatabase.php';
$conn = mysqli_connect($servername,$username,$password,$dbname);
if (mysqli_connect_errno()) { 
  die('Could not connect: ' . mysqli_connect_error()); }
$sql = "SELECT `{$obj}`, COUNT(`{$obj}`) AS counts FROM `mriTIN` 
  WHERE session = 1 AND  `id` NOT IN ('1', '2', '3', '7', '10') 
  AND  `uniqueID` IN ( SELECT `uniqueID` FROM `ppPim` WHERE 
  `rType` ={$expertType}) GROUP BY `{$obj}`;"; 
$result = mysqli_query($conn,$sql);
$response = array();
while($row = mysqli_fetch_row($result)) { 
  $json['response'] = $row[0];
  $json['counts'] = $row[1];
  array_push($response, $json); }
$sql = "SELECT uniqueID FROM `ppPim` 
  WHERE uniqueID='{$_SESSION['ppid']}' 
  AND rType={$expertType};" ;
$result = mysqli_query($conn,$sql);
if(mysqli_num_rows($result) > 0){
  $sql = "SELECT `{$obj}`, COUNT(`{$obj}`) AS counts FROM `mriTIN` 
    WHERE session = 1 AND uniqueID = '{$_SESSION['ppid']}';"; 
  $result = mysqli_query($conn,$sql);
  while($row = mysqli_fetch_row($result)){ 
    $json['response'] = $row[0];
    $json['counts'] = $row[1];
    array_push($response, $json);} }
echo json_encode($response);

The framework I used to build the bar plot is from Mike Bostock’s Let’s Make a Bar Chart d3.js example. I embedded the bar plot code into Mike’s Towards Reusable Charts example. This allowed me to recycle the bar plot function to draw the bar plots for each of the experts’ groups. Considering the aim of re-usability it is a bit contradictory that I copy-pasted the same XMLHttpRequest twice. Building this web-page would otherwise have been a truly reusable code experience. Pity. Since in explaining the code Mike did a way better job than what I could possibly do, below I will outline what I added to Mike’s code to adapt it to my needs. The complete source for the barplot function is on github.

To keep a constant ratio between the x-axis of both graph I used d3.scaleBand() which is use for categorical data instead of continuous data. Treating the numbers identifying the radio buttons as categorical variables gives a pleasant constancy and allows a straightforward comparison of the graphs. Moreover d3.scaleBand() allows flexibility because it displays all the range of possible answer even if a given item did not receive a single response (many times the extreme points in the rating scale did not receive votes, this will create graphs of different horizontal width if the numbers were treated as numbers instead of categories.). Therefore the x-axis of bar plot always ranges from 1 to 10, even if, for example, participants only gave responses in the range between 4 and 6.

The Delphi method aims to show the respondents what the community think is relevant and what is not. The salience of an item will be reflected by high scores from many respondents. To accentuate the salience of the item to the user I color coded 1) the radio buttons of the tables and 2) the bars of the bar plot. The color code I used followed the traffic-light convention. I am not sure any traffic-light convention exists, but I like calling it like this. The traffic-light convention boils down to paint 1) green the ‘Not important’ items, 2) yellow the ‘Important but not critical’ items, and 3) red the ‘Critical’ items. ‘Unable to score’ was pink because it was pink already in round 1 of the Delphi study and what is there not like in a pink bar/button?

I am not sure the way I implemented my color scale is the most efficient or effective. It was efficient in the amount of time it costed me implementing it, which boiled down to create a vector with ten colors matching the bars and index inside of the svg rect object.

var color = ["#1a9850", "#1a9850", "#1a9850", // green
             "#ffd700", "#ffd700", "#ffd700", // yellow
             "#d73027", "#d73027", "#d73027", // red
             "#f1b6da"];                      // pink

     .attr("fill",function(d,i){return color[d.response-1];})

The response of the current user was highlighted coloring the corresponding bar in blue. To determine which bar should have been blue I used the redundancy of ‘bar indexes’ in the responses array. I introduced the redundancy in the responses array with the second MySQL SELECT statement in the XMLHttpRequest. The second SELECT statement extended the array containing the responses with an extra entry of responses if the current user belonged to the given group. Given that one of the two groups had one index double, when I identified the double entry I replaced the value for the given bar in the vector color with the value for the color blue (#386cb0). I used a match in a for loop to identify the presence of the redundancy. If a match was found the slot in the ‘color’ array was updated accordingly. Using a for loop is not very elegant, but it does the job.

for(var item = 0, nItems = data.length - 1; item < nItems; item++){
        color[data[nItems].response-1]="#386cb0"; }} 

The solution above might appear intricate. The key to understanding it is to think in terms of array's indexes. The last entry in the data object contained the entry for the response of the current participant. The response field in the data object contained the index of the answer chosen by the current participant. The answer could be accessed through the last index in the object (e.g., data[nItems].response-1). Using that index to access the color array would have returned the color value for the given bar in the bar plot. Therefore, if a match was found, this index replaced the entry of the color array with the blue value (i.e., #386cb0).

All considered it was a nice project, and I am proud of the result. Adding more words to describe it might actually damage it. ;-P

Web-interface for Delphi Method III

Web-interface for Delphi Method II

Delphi method, round 2

This post describes the web-interface I built for the second round of the Delphi method study on functional magnetic resonance imaging and tinnitus. In the second round, the experts who participated in the first round saw the responses that they gave to the first round side-by-side with the responses of all the other experts. As represented in the figure above, in round 2 the Delphi interface displayed one question and two bar plots showing the distribution of experts’ responses for the given question in round 1. To keep the post short and to the point, I divided the description of the interface for round 2 into two parts. This post describes the first part, which displays the questionnaire’s items on the web page. I will describe the second part in a next post, showing how to build the bar plots to provide feedback to the participants. The web interface is here.

The display of the questionnaire items was simpler in round 2 than in round 1 because the interface presented one question at the time with the relative bar plots for the groups’ feedback. As in round 1, I presented the question and radio buttons within an HTML table. However, the code displaying the items’ text in round 2 is stripped down in comparison to round 1 because there was no section’s header.

// later on I will refer to this as 'php code for the question'
echo "<table>";
echo "<tbody><tr><td id='question' class='firstRow'>";
echo $questions[$counter][1];
echo ". </td>";
for ($item = 1; $item <= 9; $item++) {
    echo "<td> <center> <input type='radio' name='q1'"; 
    if (isset($q1) && $q1=="1") echo "checked";
    echo " value='" . $item . "'> </center> </td>";
echo "<td class='unscored'> <center> <input type='radio' name='q1' ";
if (isset($q1) && $q1=='10') echo 'checked';
echo "value='10'> </center> </td>";
echo "<td class='asterisk'> <span class='error'>"; 
echo $q1Err;
echo "    </span> </td></tr></tbody></table>"; 

To keep the question’s text and bar plots aligned I wrapped them into an HTML div. Inside of this div `wrapper', there were three div: one for the table with the questions and one for each bar plot with the responses of the experts’ groups.

<div id="wrapper">
    <div id="first">
        <!-- php code for the question -->
    <div id="second">
        <p id="firstPlot" class="titlePlot"></p>
        <svg width="300" height="150"></svg> 
    <div id="third">
        <p id="secondPlot" class="titlePlot"></p>
        <svg width="300" height="150"></svg> 

The CSS code styling the presentation is straightforward. The style sheet is in the github repository of the project.

In round 2 we included new questions and rephrased or changed words in questions which were unclear in round 1. To simplify the management of old and new questions and of the bar plot that went with them I used a two-dimensional array. I stored the question’s text in the first dimension of the array. In the second dimension I stored the indexes to identify the question in the MySQL database. This allowed keeping the order of the questions presentation coherent with the information displayed in the bar plots. So, for example, if a new question was introduced in place 13th of round 2 it was stored on slot 12 of the PHP array (which starts with 0) and with index 41 in the MySQL database since round 1 presented 40 questions. The use of the index instead of the array order allows the retrieval of the responses given to round 1 independently from the fact that the text of the question was changed, clarified, or move forward or later in the list of questions displayed in round 2. Moreover, new questions were not accompanied by a bar plot. Therefore, numeric indexes allowed a simple way of determining whether a bar plot had to be displayed or not. In fact, since only the first 40 questions had been previously answered, an index lower than 41 implied drawing the bar plots and otherwise not.

The feedback process required updating the page with new text and bar plots every response. I controlled the updating of text and plots with a counter stored in the current PHP session of the webpage. The session counter increased after the response was entered into the MySQL database. Updating of the counter was embedded in the server POST request:

    $resp1 = FALSE;
    if (empty($_POST['q1'])){ 
        $q1Err = "*"; 
    } else { 
        $q1 = test_input($_POST["q1"]); $resp1 = TRUE;}
    if ($resp1){
        $qNn = 'q' . ($questions[$counter][0]);
        $sql = "UPDATE `{$tableName}` SET `{$qNn}` = '{$q1}' 
            WHERE uniqueID =  '{$_SESSION['ppid']}' AND session = 2";
        if (!mysqli_query($conn, $sql)){ 
            die('Error: ' . mysqli_error($conn)); }
        $q1 = $q1Err =  "";
        if (!isset($_SESSION['count'])) {
            $_SESSION['count'] = 0;
        } else {$_SESSION['count']++;}
    } else { 
        $requiredFields = "* required fields"; }// END: if ($resp1

Sanity checks

The second round of the Delphi questionnaire is useless if a participant does not receive feedback from his/her previous session. Therefore, the first sanity check prevented participants from continuing without a unique identifier. A warning text displayed on a red background advised users with no identifier that they should have gotten a unique identifier before continuing. The PHP $SESSION also stored the unique identifier.

if (!array_key_exists('ppid',$_SESSION) || empty($_SESSION['ppid'])){
    echo "<p>
      <span style='background-color:#d73027; color:white; font-size:x-large;'>
    You do not have an ID to submit the responses!! Please go to this ";
    echo "<a href='http://localhost/~mp/part2/delphi2.php'>";}
    echo "page</a>, fill in your first and last name, and get an ID!";
    echo "</span></p>";

Moreover, participants with a unique identifier were matched on the database containing the responses from round 1 to determine whether their responses were present or not:

$ppName = $ppSurname = "";
$ppNameErr = $ppSurnameErr = "";
$requiredFields = "";
  $ppResp  = $surnameResp = FALSE; 
  if (empty($_POST['ppName'])) { $ppNameErr = "*"; } 
  else { $ppName = test_input($_POST["ppName"]); $ppResp = TRUE;}
  if (empty($_POST['ppSurname'])) { $ppSurnameErr = "*"; } 
  else { $ppSurname = test_input($_POST["ppSurname"]); $surnameResp = TRUE;}
  $_SESSION['ppid'] = $ppID;
  if( $ppResp && $surnameResp) {
     require '../includeDatabase.php';
     $conn = new mysqli($servername, $username, $password, $dbname);
     if ($conn->connect_error) {die("Connection failed: " . 
         $conn->connect_error);	}
     $sql = "SELECT uniqueID FROM `{$tableName}` 
         WHERE uniqueID = '{$ppID}'";
     $result = $conn->query($sql);
     if ($result->num_rows > 0) {
        $sql = "INSERT IGNORE INTO mriTIN (`uniqueID`, `session`) 
                VALUES ('{$ppID}', 2);";
        if ($conn->query($sql) === FALSE) { echo "Error: " . 
             $sql . "<br>" . $conn->error; }
	header("Location: http://localhost/~mp/part2/delphi2_2.php");
      } else {
	echo "<p style='background-color:#d73027; 
              color:white; font-size:x-large;'>We could not find a user 
              matching " . $ppName . " " . $ppSurname . "<br>";
	echo "If you did participate in Round one of this questionnaire 
              then you might be using an ID we do not recognize. Please 
              get in touch with Paolo (" ;
	echo "<a href='mailto:p.toffanin@umcg.nl'>p.toffanin@umcg.nl</a>) 
              to sort out the issue. Apologies for the inconveniences.</p>";
  } else {$requiredFields = "* required fields";} // END: if( $ppResp && ...
} // END: if ($_SERVER["REQUEST_METHOD"] == "POST") {

Participants for whom a unique identifier was not found were asked to mail me for help or to complete round 1 before completing round 2.

Sanity checks include also form validation. Here I will not discuss form validation since I treated it in the post for round 1 (see here). Moreover, since most of this form’s inputs are radio buttons, form validation is superfluous.

In the next post, I will address the Javascript code for the bar plot, the XMLHTTP request retrieving the data and MySQL commands to retrieve the counts for the responses of each question.

Web-interface for Delphi Method II

My favorite pizza dough

Because lately I have been writing too often about serious stuff, today I will share the result of another type of experiments: the recipe of my favorite pizza dough. I have been experimenting making my own pizza dough for the last three years and I believe I have now found a recipe which yields a good result. The recipe distills and combines the wisdom of three very positive influences, Dan Leppard, Emmanuel Hadjiandreou, and il cucchiaio d’argento.

First of all, I must warn that the procedure takes a bit of discipline but only because timing is always a sensitive matter when baking. However, the procedure in itself is simple and relatively fool proof, in the sense that it is easy to fix if something goes wrong: add a bit of flour or water depending on the thickness of the dough. Moreover, the ingredients are easy to find. I divided the procedure into three steps, each requiring (at least) an hour of waiting/rising time before moving to the next step. Usually, I start Sunday after lunch, say around 13 o’clock, and I take the first pizza out of the oven at 17:30.


  • 350 ml water @ room temperature
  • 1 teaspoon yeast
  • 200 g Bread/strong/hard flour (e.g. at least 14% gluten protein)
  • 400 g all purpose flour
  • 50 g polenta flour
  • 20 g of lard (melted bacon fat)
  • 1 teaspoon salt
  • 1 teaspoon sugar

Step one

Dissolve the yeast in the water. In a bowl mix all flours and salt, add to the water-yeast mixture and then include the lard. When all ingredients are homogeneously mixed cover the bowl and let rest for 10 minutes. Passed the ten minutes mix or knead for 10 seconds and cover again. After repeating this procedure three times, let the dough rest covered for 1 hour (or longer). For covering the bowl I like to use a plastic shower cap. A wet/humid cloth or lid will do as well. However, the dough will dry if the towel dries or if the lid lets air through. As a consequence a film/skin like layer will form on the surface of the dough, which is not very pleasant. Also, a perfect spot to place the dough to rise is the top of a drier, if you have a wash needing to get dry.

Optional kneading step

Since I love kneading the dough, at this stage but before letting the dough to rest for an hour, I knead it for a while. The amount of time I knead varies from 0 seconds to 15 minutes. If I do knead I spread a little olive oil on my palms and fingers and knead until it the dough absorbs it. I repeat this procedure 0 or 6 times depending on my ‘kneading mood’. I guarantee I cannot say I notice a difference in the elasticity of the dough from when I do the kneading or when I skip it. However, I do enjoy the feeling of handling the malleable dough with greasy hands. This kneading step does somehow improve your dough because by adding oil the pizza will become more crunchy when baked than when less or no (extra) oil (or fat) is added. However, skipping this step has fewer disastrous consequences than, for example, forgetting the salt.

Step two

Extract the dough from the bowl and cut it into four approximately equal parts (or into five parts if you like smaller pizzas). Roll the parts into balls and lay to rest on a top sprinkled with flour. Cover the balls with a humid cloth. The length of the interval I leave the dough to rest/rise depends on what I have to do and varies from one-half hour to two hours if, for example, I have to go and do the weekly groceries. Forming the balls before letting the dough rise at this step facilitates making the pizza in the shape of a circle later on when flattening the dough into a pizza.

Step three

I like the pizza best when it has a thick crust. Getting a thick crust is only a matter of flattening the dough in the right manner. I achieve the best result using this procedure: on a floured surface I flatten the balls of pizza dough starting from the center pushing toward the border using the tip of my fingers (i.e. the distal phalanges). Once I pressed down I rotate the ball a bit and I flatten the dough again. I keep rotating and flattening until the pizza has the desired size and thickness. This step takes a bit of practice, but if you fail miserably you can always renounce to the crust and use the rolling pin.
Garnish with your favorite toppings and bake for 10-15 at the maximum temperature your oven can do (I assume a maximum temperature of 300 degrees, if your oven can go higher the baking time is shorter). I usually put a pizza in the oven every 5 minutes so that we get a freshly baked pizza almost immediately after we finished eating the current one.

That is it. Of course there is room for variations but I will leave those for a later revision.

My favorite pizza dough

Streamgraph in R [final]

This post is an update on the previous post translating Byron and Wattenberg’s streamgraphs algorithm into R. Byron and Wattenberg’s algorithm produces beautiful streamgraphs with the synthetic data produced by their streams generator. However, the implementation yields an ugly streamgraph when applied to data which might not be as wiggly as the synthetic ones. In the attempts I made I got very peaky wiggles, not smoothed and irregular. In short the graphs did not transmit the idea of a stream, but of a blurry blob or a peaky primitive bat (the wooden club, not the animal, that would be cool!). In this post I bring-up some points to bear in mind when producing a streamgraph. Continue reading “Streamgraph in R [final]”

Streamgraph in R [final]

Streamgraphs in base::R [e.III]

This is the third post on streamgraph in R. After a simple introduction on how to generate a streamgraphs and an example with actual data it was time for a more general implementation to the creation streamgraphs using R. Continue reading “Streamgraphs in base::R [e.III]”

Streamgraphs in base::R [e.III]

Multiple mediation: extracting output

Lavaan’s output is copious. Especially when fitting a complex model, lavaan’s output is literally a mine of information. Finding what is relevant is not always easy, therefore I will try to describe a way to summarize lavaan’s summary output. Continue reading “Multiple mediation: extracting output”

Multiple mediation: extracting output