Filter text before it is displayed. Find and replace invalid charaters from cms content using JavaScript and ASP

The content for the website is generated by the staff and put into the site using the content management system. Much of this content is created in a Word Processing program such as Microsoft Word. In many cases, the content is not formatted correctly for the web. While the cms does a good job of filtering out the Word Document markup, there are still some charaters that get through the filter, but don’t display properly on the web. These  characters show up as odd symbols such as diamonds and squares. I have created a filter to run the content through before it is displayed, finding the invalid characters and replacing them with html character codes.

 

Due to the restraints of the system I am working in, I decided to go with a vbscript dictionary object to hold an array of characters that were not displaying properly in the browser. I created an api that could be included into any page, and used an algorithm to detect and replace the pesky invalid characters.

‘——————————
‘_____________________________
‘ function to replace extended asci chars with the html equivalent ‘***************************
Function replace_characters(inString)
newstring = inString
dim lamechars
Set lamechars = CreateObject _
(“Scripting.Dictionary”)
lamechars.Add “–”, “–”
lamechars.Add “®”, “®”
lamechars.Add “’”, “’”
lamechars.Add ““”, ““”
lamechars.Add “””, “””
lamechars.Add “—”, “–”
For Each character in lamechars
newstring=Replace(newstring, character, lamechars(character))

Next
Response.Write(newstring)
End Function
‘**************************
‘ end function to replace extended asci chars with the html equivalent ‘—————————

JQuery Mobile Single Page Template

This is the standard page template you should start with on a project:

 <!DOCTYPE html> 
<html> 
<head> 
<title>Page Title</title> 
<meta name="viewport" content="width=device-width, initial-scale=1"> 
<link rel="stylesheet"
 href="http://code.jquery.com/mobile/1.0/jquery.mobile-1.0.min.css" /> <-----this will need to be updated to the latest cdn link
<script type="text/javascript" 
src="http://code.jquery.com/jquery-1.6.4.min.js"></script> <-----this will need to be updated to the latest cdn link
<script type="text/javascript" 
src="http://code.jquery.com/mobile/1.0/jquery.mobile-1.0.min.js"></script> <-----this will need to be updated to the latest cdn link
</head> 
<body> 
<div data-role="page"> 
<div data-role="header"> 
<h1>Page Title</h1> 
</div> 
<!-- /header --> 
<div data-role="content"> 
<p>Page content goes here.</p> 
</div> 
<!-- /content --> 
<div data-role="footer"> 
<h4>Page Footer</h4> 
</div> 
<!-- /footer -->
</div>
 <!-- /page --> 
</body> 
</html> 

Looking at the PHP Form Post Object

I use php to talk with MySQL db server. It comes in handy for making apps. Here are the code variations for receiving POST variables from a client request and ,for debug, a quick list of the variables, names and values. Using this: print_r($_POST); this gives a breakdown of the array holding the data from the post variables…. Array ( [signindate] => 11/4/2011 [signintime] => 15:33 PM [nameofchild] => dev en [parentsignature] => don e [in] => Sign In ) to access an element of the array I use a couple methods.. to get them one by one  by name so I can put them into the database.. $myformelement=$_POST[‘form element field html id name’];

Now that the records are correctly inserting into database, my next step is to  use some ajax to call the php behind the scenes while leaving my webapp ui view intact.

  • Check if a person is selected
  • Create an XMLHttpRequest object
  • Create the function to be executed when the server response is ready
  • Send the request off to a file on the server

Add a Zero (0) in front of numbers below ten (10)

In certain situation I need to add a zero in front of numbers below ten. It could be used for money, .05 and for time :05 and I’m sure there are other situations where this could be useful.

// add zer0 in front of minutes below ten 10
function pad(n) {if (n < 10) return “0” + n; return n;}

// use the pad function
for(var minute=0; minute<50;minute=minute+5){
$(“#minutes”).append(“<option value='”+pad(minute)+”‘>”+pad(minute)+”</option>”);
}

Web Form Security Practices

We have had a lot of attacks to our web forms here at SVC. I am the paperless applications developer, so the task of keeping them safe and secure falls into my daily operations. An un secure form can let an attacker do many things to ruin someones day, such as sending unsolicited emails to all the staff and students. Some of the common practices to keep the “bots” from breaking into a web form are:

CAPTCHA
This technique provides the user with a series of image “keys”  that they must “unlock” before they can submit the web form. This is a  good way to slow down the bots, but since they don’t get tired, and can hammer away at the form many times each second, they can eventually bypass this technique. Another problem with captcha, and a very important one, is regarding the user experience. Some captchas are impossible to read, and take several tries for a normal user to get correct. This can really damper the user experience and create a situation where the user will just leave instead of taking the time to keep trying to unlock the captcha keys. As stated by the w3, “This type of visual and textual verification comes at a huge price to users who are blind, visually impaired or dyslexic. Naturally, this image has no text equivalent accompanying it, as that would make it a giveaway to computerized systems. In many cases, these systems make it impossible for users with certain disabilities to create accounts, write comments, or make purchases on these sites, that is, CAPTCHAs fail to properly recognize users with disabilities as human.”

So, what else can I do to help keep the bots from submitting forms?

1: Try to make sure that the form is submitted from the form page, because the bots usually submit directly to the form action. Put some kind of trigger in the form submit actions that look for a referring address or a session variable.

In classic ASP we have the Request.ServerVariables collection that can get some important information, and if the form data has not come from the form page, we know its a bot, and can secretly send a security alert to the admin, while not letting the bot know its been busted.

Request.ServerVariables("URL"), SCRIPT_NAME, and PATH_INFO

should all contain the form’s address, and should show that the form was submitted from the proper location.

Request.ServerVariables("SERVER_NAME")

should be checked for the proper domain, indicating whether the form was properly posted from my server. I could create an include to be used before each form submission to check for coming from my server.

These could be put into hidden fields and submitted with the form, then checked for validity before processing and submitting the form. Also a session variable might be a good way to track these from form submit to data capture.

In the form capture logic, we could look for Request.ServerVariables(“HTTP_REFERER”) to ensure that the data has come from our form page.

So, lets try some of this junk and see if it helps out.
First, lets set a session of “formuser” with a value of Request.ServerVariables(“URL”) when the user accesses the form, and then check for that session in the form submit logic. If there is no session, then this data did not come from our form.

Creating dynamic selects using Javascript and JQuery

I need to have a select form element, with the options generated from dynamic content that could come from a database or an array, with an unknown number of options. Then, I need to create two more selects, with all of the options except the ones selected in the previous select. For example, a user must choose their first, second, and third choices for college.

Lets say we have a list of colleges and their abbreviations:

<input type=”checkbox” name=”scholarshipschools” value=”CWU” title=”Please select a college.” /> Central Washington Univ, Ellensburg <br />
<input type=”checkbox” name=”scholarshipschools” value=”EWU” /> Eastern Washington Univ, Cheney <br />
<input type=”checkbox” name=”scholarshipschools” value=”SVC”  /> Skagit Valley College <br />
<input type=”checkbox” name=”scholarshipschools” value=”ESC” /> The Evergreen State College, Olympia    <br />
<input type=”checkbox” name=”scholarshipschools” value=”UWB” /> Univ of Washington, Bothell <br />
<input type=”checkbox” name=”scholarshipschools” value=”UWS” /> Univ of Washington, Seattle <br />
<input type=”checkbox” name=”scholarshipschools” value=”UWT” /> Univ of Washington, Tacoma <br />
<input type=”checkbox” name=”scholarshipschools” value=”WSU” /> Washington State Univ, Pullman <br />
<input type=”checkbox” name=”scholarshipschools” value=”WWU” /> Western Washington Univ, Bellingham

 

The checkboxes don’t allow the user to choose what ones would be their first, second and third choices. Also, doesn’t let the user know that they are limited to choosing three. I have been asked to redesign this form so that the user has a better interaction with the page.

I don’t want to type the names of the schools and their abbreviations more than once while I develop this, so the first thing I want to try is to put them into an array, then I can use them when, where and in whatever container I wish. (like changing from checkboxes to selects).

 

var thecollegeabbr = [“CWU”,”EWU”, “SVC”, “ESC”, “UWB”, “UWS”, “UWT”, “WSU”, “WWU”];

var thecolleges = [“Central Washington Univ, Ellensburg”, “Eastern Washington Univ, Cheney”, “Skagit Valley College”, “The Evergreen State College, Olympia”, “Univ of Washington, Bothell”, “Univ of Washington, Seattle”, “Univ of Washington, Tacoma”, “Washington State Univ, Pullman”, “Western Washington Univ, Bellingham”];

I would like for the college name to have its abbreviation attached to it, rather than have them in two separate arrays. Using the information from my previous post on JavaScript Array Best Practices, I see that I could streamline this a little, and make a more efficient 2-dimensional array. I will use a for-if loop to push the abbreviation and title into a two dimensional array.

var thecollegechoices=[];
// combine the abbreviation and the title into a two dimensional array…
for(i=0;i<thecollegeabbr.length;i++){thecollegechoices.push([thecollegeabbr[i], thecolleges[i]]); }

Now that I have an array of data to populate the selects, I will create an algorithm to  put the data into the “First Choice” selector:
// now put the data into a select element
for(i=0;i<thecollegechoices.length;i++){
$(“#firstcollegechoice”).append(“<option value='”+ thecollegechoices[i][0] +”‘>”+ thecollegechoices[i][1] +”</option>”);
} // end for loop to put the data into a select element

screen shot of dynamic selectThe results are very effective, as I now have a select that was dynamically populated using the array of data that I was provided. The next step is where this really starts to get fun. Now I need to create a “Second Choice” selector using all of the choices from the first one, except for the one that was chosen by the user in the “First Choice” selector. This will need an algorithm that is a bit more complex, and will be triggered by the event of the user choosing the first choice.

From my previous article about using JQuery to handle events, I know that I can access the onchange event handler and give it a method that will run my function when the user changes the value of the select. I will build the algorithm into this method as so:

  • Get the value of the college that the user has chosen.
  • Rebuild my array of “thecollegechoices”, eliminating the option that was previously chosen.
  • Populate the options of the “Second Choice” selector using the new data that has been  put into thecollegechoices array.

Here is a snippet of what this looks like now…

$( ‘#firstcollegechoice’ ).on(“change”, function(){
var thecollegechoices=[];
for(i=0;i<thecollegeabbr.length;i++){
// check here to see if the choice is used in the first select.
if(thecollegeabbr[i]!=$(“#firstcollegechoice :selected”).val())                                {thecollegechoices.push([thecollegeabbr[i], thecolleges[i]]);}
}

for(i=0;i<thecollegechoices.length;i++){$(“#secondcollegechoice”).append(“<option value='”+ thecollegechoices[i][0] +”‘>”+ thecollegechoices[i][1] +”</option>”);}

}); // end first choice change functions

 

Dang, this is lookin good. I now have a second choice selector with all the options except for the one that was chosen by the user in the first selector.

It seems like an easy path to our goal, a “Third Choice” selector with all of the options except what was chosen in one and two. Then we are left with some ui clean up to ensure a good user experience. I like to use responsive disclosure to guide the user through a form whenever possible. For this instance, I will disable the second and third choice drop-downs until they are needed, so the user will easily see what they need to do.

———————————– here is the dev code ———————————–

<select name=”scholarshipschools” id=”firstcollegechoice” ><option value=””>First Choice</option></select>
<select name=”scholarshipschools” id=”secondcollegechoice” disabled=”disabled”><option value=””>Second Choice</option></select>
<select name=”scholarshipschools” id=”thirdcollegechoice” disabled=”disabled”><option value=””>Third Choice</option></select>

<script>
// make an array to hold the college choices.
var thecollegeabbr = [“CWU”,”EWU”, “SVC”, “ESC”, “UWB”, “UWS”, “UWT”, “WSU”, “WWU”];
var thecolleges = [“Central Washington Univ, Ellensburg”, “Eastern Washington Univ, Cheney”, “Skagit Valley College”, “The Evergreen State College, Olympia”, “Univ of Washington, Bothell”, “Univ of Washington, Seattle”, “Univ of Washington, Tacoma”, “Washington State Univ, Pullman”, “Western Washington Univ, Bellingham”];
var thecollegechoices=[];

// combine the abbreviation and the title into a two dimensional array…
for(i=0;i<thecollegeabbr.length;i++){thecollegechoices.push([thecollegeabbr[i], thecolleges[i]]); }

// now put the data into a select element
for(i=0;i<thecollegechoices.length;i++){$(“#firstcollegechoice”).append(“<option value='”+ thecollegechoices[i][0] +”‘>”+ thecollegechoices[i][1] +”</option>”);}

$( ‘#firstcollegechoice’ ).on(“change”, function(){
//$(“#debugzone”).html($(this).val() + ” chosen now build the second select<br />”);
$(“#secondcollegechoice”).html(“<option value=”>Second Choice</option>”).removeAttr(“disabled”);$(“#thirdcollegechoice”).html(“<option value=”>Third Choice</option>”).attr(“disabled”, “disabled”);
var thecollegechoices=[];
for(i=0;i<thecollegeabbr.length;i++){
// check here to see if the choice is used in the first select.
if(thecollegeabbr[i]!=$(“#firstcollegechoice :selected”).val()){thecollegechoices.push([thecollegeabbr[i], thecolleges[i]]);}
}

for(i=0;i<thecollegechoices.length;i++){$(“#secondcollegechoice”).append(“<option value='”+ thecollegechoices[i][0] +”‘>”+ thecollegechoices[i][1] +”</option>”);}

});

$( ‘#secondcollegechoice’ ).on(“change”, function(){
//$(“#debugzone”).html($(this).val() + ” chosen now build the third select<br />”);
$(“#thirdcollegechoice”).html(“<option value=”>Third Choice</option>”).removeAttr(“disabled”);
var thecollegechoices=[];
for(i=0;i<thecollegeabbr.length;i++){
// check here to see if the choice is used in the first select.
if(thecollegeabbr[i]!=$(“#firstcollegechoice :selected”).val() && thecollegeabbr[i]!=$(“#secondcollegechoice :selected”).val()){
thecollegechoices.push([thecollegeabbr[i], thecolleges[i]]);
}
}

for(i=0;i<thecollegechoices.length;i++){$(“#thirdcollegechoice”).append(“<option value='”+ thecollegechoices[i][0] +”‘>”+ thecollegechoices[i][1] +”</option>”);}

});

</script>
<!– end new content dev area –>