Why does REST API seem so hard to understand?

Introduction

Early in the course of my software development career, I realized much later that I had been writing what would qualify to be REST API, or would qualify to be so with minimal refinement or enhancement.

I had before written so many endpoints in the backend of web-applications, and AJAX/Fetch calls in Javascript for consuming the endpoint resources (by making HTTP requests and receiving HTTP responses). Note: I have never used Axios for making requests, but that shouldn't matter anyway.

Obviously, the concept of HTTP request/response would be applied in all the situations I have gone through as highlighted in the preceding paragraph. In hindsight, I believe that such experience should give anyone the competence to write REST APIs.

Unfortunately, whenever I looked up tutorials or write-ups on REST API (also known as RESTful API), many of them would look so intimidating and confusing instead. I had found it rather difficult to firmly grasp the concept of REST API, until I understood that it is just any other API, but following certain design rules. So I decided today to share my experience just in case it may help others In the same situation where I was back then.

Wait a minute ... up to his point, I have been writing in the "third person". All above was not necessarily my personal story or experience. It was about a few people whom I had interacted with, and I learnt from them that REST API could be challenging to understand. I only represent them by simply summing up important points from their stories. By extension, REST API could be challenging to others too.

Everybody probably knows that REST stands for Representational State Transfer. That is the very easy part – maybe – but what does that mean exactly? This is where the confusion starts.

Definitions usually seem too complicated, unless you directly relate them to practical work that you have done by yourself. This makes sense because theoretical software development effort without a practical touch has very little to yield. Read, read and read – but for as long as you don’t put what you have read into practice, you will gain so little out of your effort, if not nothing at all.


Back to REST API, but let’s start with API.

API (Application Programming Interface) is a software intermediary that allows two applications to talk to each other and share resources (data or functionality). An API offers services to other software to enhance the required functionalities. It can offer resources to external third-party developers. Simple, isn’t it?

The following definition according to IBM makes it clearer: "At the most basic level, an API is a mechanism that enables an application or service to access a resource within another application or service. The application or service doing the accessing is called the client, and the application or service containing the resource is called the server."

An example of API would drive the point home, though it would make more sense if you do develop software yourself.

Take the example of Google Maps. When you search for the location of a place using Google Maps, your request goes to the appropriate endpoint in the Google Maps API. The endpoint processes your request and delivers a response back to you. The response in this case is some data (not HTML), and most likely in form of JSON or XML.

An API uses HTTP methods/verbs (POST, GET, PUT or DELETE) for communication in line with CRUD operations respectively. In general, POST creates a new resource, GET reads a resource, PUT updates a resource, and DELETE deletes a resource.

The concept of "idempotent" is important for REST API as we shall mention later. Just to note: GET, PUT and DELETE methods are idempotent (has no additional effect whether you run it once or repeat it N times). POST method is not idempotent (it can have additional effect of creating N resources if run N times).

To put it another way, GET, HEAD, OPTIONS are safe and idempotent methods whereas PUT and DELETE methods are only idempotent. POST and PATCH methods are neither safe nor idempotent.

An HTTP request consists of method/verb, URI, HTTP version, header, body; whereas an HTTP response consists of  response status code, HTTP version, header, body. This is important for the communication.


Now, what is REST API?

REST API is just an API that follows a set of rules or styles for applications and services to communicate with each other using HTTP requests and responses while performing the standard database CRUD operations.

RESTful web services are services that follow REST architecture. REST stands for Representational State Transfer and uses HTTP protocol (web protocol) for implementation. These services are lightweight, provide maintainability, scalability, support communication among multiple applications that are developed using different programming languages. They provide means of accessing resources present at server required for the client via the web browser by means of request headers, request body, response body, status codes, etc.

The REST Server provides access to REST resources (text files, HTML pages, images, or any other dynamic data) whereas the REST client consumes (accesses and modifies) these resources. Every resource is identified globally by means of a URI.

The meaning of idempotent is that even after calling a single request multiple times, the outcome of the request should be the same. While designing REST APIs, we need to keep in mind to develop idempotent APIs. This is because the consumers can write client-side code which can result in duplicate requests intentionally or not. Hence, fault-tolerant APIs need to be designed so that they do not result in erroneous responses.

REST is simply a set of architectural constraints (not a protocol or a standard) that can guide API developers.

REST API must conform to the following constraints/criteria: statelessness, cacheable, decoupled, layered system, and uniform interface. This is where the difficulty in understanding REST API lies.

Therefore, if my API can conform to the above constraints, then I would call it a REST API. This means that the gist of understanding REST API is in the constraints. Let’s try to look at them one by one.

• Statelessness: Each request is independent of the other; a request knows nothing of any other request. REST APIs do not require any server-side sessions. Server applications aren’t allowed to store any data related to a client request.
• Cacheable: Caching helps improve performance and address the constraint of statelessness. When possible, resources should be cacheable on the client or server side
• Decoupled: Client and server are decoupled/detached from each other. The client knows only the URI, and the server passes the response via HTTP without making any client changes.
• Layered : REST APIs should be modular and more flexible to achieve scalability. It should be layered in the way it organizes servers. Requests and responses go through different layers. REST APIs need to be designed so that neither the client nor the server can tell whether it communicates with the end application or an intermediary.
• Uniform Interface: All API requests for the same resource should look the same, no matter where the request comes from. The REST API should ensure that the same piece of data, such as the name or email address of a user, belongs to only one uniform resource identifier (URI).

In addition to the above (and as mentioned earlier), REST APIs are lightweight, easily-maintainable, scalable, and platform-independent. By now, if you are familiar with HTTP, then most likely understanding REST API has become easier.

The Role of JSON in Web-Application Development

Introduction

JSON stands for Javascript Object Notation. It is a text-based format for representing structured data, and is commonly used for data interchange over a network (between client and server).

If you are a web-developer, then you have undoubtedly used JSON. If not, you must immediately learn about JSON before time catches up with you.

JSON is a string, which follows the Javascript Object syntax. For example:

Javascript object - {firstname: “Richard”, lastname: “Orama”}
JSON - '{“firstname”: “Richard”, “lastname”: “Orama”}' //this is a string, so it should be in single quotes overall

Please note the obvious “name: value” pairs and the difference in quotes around the names above between JSON and Javascript object. The names in JSON are double-quoted, whereas it is not the case with Javascript object.

As a data interchange format, JSON can be used for sending data from client to server for processing, as well as sending data from server to client for display. Bring back memories of HTTP request and response immediately.

JSON can be nested (and deeply-nested too), so it offers a way to represent complex data, with many practical use-cases.

The creation of a JSON string is a serialization process. To access JSON in Javascript, one needs to convert it to native Javascript object using the Javascript JSON object (deserialization). To handle JSON on the server, every programming language has appropriate libraries for doing so.

In the context of data storage, serialization is the process of translating data structures or object state into a format that can be stored (for example, in a file or memory buffer) or transmitted and reconstructed (deserialized) later. Converting data to JSON is a serialization process. Reading (parsing) the data later is deserialization.

It is important to master how to read (parse) and create JSON on the client and server sides. I will now give you some examples using Flask backend and Vanilla Javascript.


Client

Create JSON: JSON.stringify(javascriptObject)
Read JSON: JSON.parse(jsonString). We can also convert a JSON string to Javascript object using the json() method of the Response interface.


Server

Create JSON: json.dumps(dictObject), or specifically for Flask, you can use jsonify(dictObject)
Read JSON: request.get_json(), but mimetype must be application/json; if query string was used, then request.get.args(); if formdata was used, then request.form.get()


Snippet (Client – Create and Read JSON)

let data = {firstname: “Richard”, lastname: “Orama”}; //javascript object
fetch(url, {
		method: 'POST',
		headers: {
			'Content-Type': 'application/json',
			'X-Source-From-Fetch': 'True',
		},
		body: JSON.stringify(data), //CREATE JSON (serialize)
	})
	.then(response => response.json()) //READ JSON //de-serialize/parse json to javascript object, can now access using response["key"] or response.key
	.then(data => {
	//do something
	})
	.catch((error) => {
		console.error('Error:', error);
	});

Snippet (Server – Read and Create JSON)

request_data = request.get_json() #READ JSON
if ‘firstname’ in request_data:
firstname = request_data[‘firstname’]
if ‘lastname’ in request_data:
lastame = request_data[‘lastname’]
…
return jsonify({‘result’: ‘OK’}) #CREATE JSON - could use the generic Python json.dumps()

If you are programming in any other backend language, it should still be pretty easy to follow along from the above examples.

How important are Data Structures in Software Development and Data Science?

Someone asked me: when do we apply data structures in software development and data science?

This basic question is important for many reasons, but most importantly, it makes one appreciate the all-so-important concept of data structures in a practical way. My answer to the above question was that I use data structures all the time I am coding the backend in a web-application or a data science project. Perhaps the answer could easily be that I use it all the time I am coding.

Today, I want to talk about the importance of data structures in software development and data science based on my day-to-day work. I want to expose, using sample codes, instances where I am using data structures.

First off, currently I do backend development and data wrangling, analytics, and reporting mostly using Python. Previously, I developed applications using Java, and even Groovy. I only used C++ as part of my educational projects. I remember writing a text editor then for a statistical analysis package using C++, but that was more like a toy project. In the earlier stages of my career, my colleagues and I would also sometimes question the importance of data structures (and algorithms) in practical software development. In hindsight, I wonder why.

In this post, I will demonstrate that data structures are used virtually every time you are writing code for a project. I will illustrate using Python/Flask, but this specificity is inconsequential. The principle should apply across the board.

For you to comfortably grasp this submission, you should be familiar with data types and data structures (at least the basic ones) in Python or any other language.

Python has the following primitive data types: int, float, string, boolean. A date in Python is not a data type of its own, but we can import a module named datetime to work with dates as date objects.

Data Structures are built around the primitive data types, and they provide a means for organizing and efficiently accessing data in a program.

In Python, we have the following standard Data Structures, and they are all containers:

1. string (sequence, immutable), e.g ‘richard orama’
2. list (sequence, mutable, heterogeneous), e.g. [2, 6, ‘richard’, (1, 2)]
3. tuple (sequence, immutable, heterogeneous), e.g. (10, 20, 30)
4. dictionary (collection, mutable, key/value pairs), e.g. {‘name’: ‘orama’, ‘programming_languages’: [‘python’, ‘java’, ‘javascript’], ‘databases’: (‘mysql’, ’postgresql’)}
5. set (collection, mutable, unique members, unordered) {11, 55, ‘kampala’}

Being containers, these data structures can contain one or a combination of primitive data types or another container. For example, a list can have membership of int, string, tuple.

Other Python data structures can also be constructed: queue, stacks, etc - but we shall not reach that extent.

While developing a web application, I find that I use lists and dictionaries a lot (if not all the time). I use tuples less often, but still considerably. I use sets less often, but in ways that should not be ignored because when I use a set, there is no other option that I can use in that situation other than a set.

I will now show you some examples of when I use data structures so that you can appreciate their use cases. To take the point home, I am picking random samples of Flask functions or endpoints just to demonstrate that data structures are applied everywhere. The snippets should be self-explanatory and informative enough even if you develop in another language.

@app.route('/select_districts', methods=['GET'])
@exception_handler
def select_districts():
    """
    Select all the records useful for populating drop-downs
    """

    district_list = District.query.filter(District.enabled == 1).with_entities(
        District.id, District.record_name).all()
    list_zip = list(zip(*district_list))
    district_list_id = list_zip[0]
    district_list_name = list_zip[1]
    result = {'districtListId': district_list_id, 'districtListName': district_list_name}
    escape_exclusions = ['aHtml']
    return jsonify({k: v if (not isinstance(v, str) or k in escape_exclusions) else escape((v)) for k, v in result.items()})

In the above snippet:

• district_list and list_zip are lists
• district_list_id and district_list_name are list elements
• result is a dictionary
• the return value (json) is a dictionary.
• zip(district_list) is a zip object, which is an iterator (in this case list) of tuples

def read_data(report_name, values_matrix, aggfunc_matrix):

    sql_fields = request.args.get('sqlFields')
    sql_from = request.args.get('sqlFrom')
    dimensions = []

    dimensions = map(lambda x: 'T.'+x.strip(),
                     session.get('report_dimensions').split(',')[1:]) #leave out None

    # converting map to list works only once, subsequent times it becomes empty list
    dimensions_list = list(dimensions)

    sql_text = f"select {sql_fields} from {sql_from} where 1"

    if not validate_sql(sql_text, 'select'):
        return (None, None)

    if session['DBMS'] == 'MySQL':
        # mysql
        with pymysql.connect(host='localhost', port=3306, user='grails', password=os.environ.get('PASSWORD_MYSQL'), db='nomad1') as dbConnection:
            df = pd.read_sql(sql_text, dbConnection)
    elif session['DBMS'] == 'SQLite':
        sql_text = sql_text.replace("concat(c.textfield2, ' ', c.textfield1)", "c.textfield2 || c.textfield1")
        # Create a SQL connection to our SQLite database
        sqlite_file = os.path.join(basedir, app.config['SQLITE_FOLDER'], 'database.db')
        with sqlite3.connect(sqlite_file) as dbConnection:
            df = pd.read_sql(sql_text, dbConnection)

    return (df, [s.replace('T.', '') for s in dimensions_list])

In the above snippet:

• request.args and session are dictionaries, ...

@logging
def report_matrix(**kwargs):
    ''' matrix report
    '''

    df = kwargs['df']
    report_group_rows = kwargs['report_group_rows']
    report_group_cols = kwargs['report_group_cols']
    values_matrix = kwargs['values_matrix'].replace('t.', '')  # remove 'T.'
    aggfunc_matrix = kwargs['aggfunc_matrix']

    # could use generator instead of list comprehension - simply change [] to () - its more efficient for large datasets
    index = [c for c in report_group_rows if c != 'None']
    # make unique. cannot use set since it is unordered
    index = list({x: 'Dummy' for x in index if x != 'None'})

    columns = [c for c in report_group_cols if c != 'None']
    # make unique. cannot use set since it is unordered
    columns = list({x: 'Dummy' for x in columns if x != 'None'})

    df['Join_Quarter'] = df['Join_Date'].dt.quarter
    columns[-1] = 'Join_Quarter' #replace last with this

    #get pivot for ach group, but remove unnecessary column totals (uing iloc[:, :-1])
    report = df.groupby(index[0]).apply(lambda sub:
        sub.pivot_table(
            index=index,
            columns=columns,
            values=[i.strip() for i in values_matrix.split(',')][0],
            aggfunc=aggfunc_matrix, #[np.sum, len], # aggfunc_matrix,
            fill_value=0,
            dropna=True,
            margins=True,
            margins_name=f'SubTotal {sub.name}'
        ).iloc[:, :-1] #remove pivot column subtotals
    ).sort_index(axis=1)
    #row summaries
    number_levels = report.index.nlevels
    row_max = ['', 'Max']
    row_min = ['', 'Min']
    row_total = ['', 'Total']
    for _ in range(number_levels-2):
        row_max.append('')
        row_min.append('')
        row_total.append('')

    report1 = report[~report.index.get_level_values(1).str.contains('Total')] #exclude totals
    report.loc[tuple(row_max)] = report1.max()
    report.loc[tuple(row_min)] = report1.min()
    report.loc[tuple(row_total)] = report1.sum()
    #column summaries
    report['ColumnMax'] = report.max(axis=1)[:-3] #ignore last three emelemts of series
    report['ColumnMin'] = report.drop(columns=['ColumnMax']).min(axis=1)[:-3] #ignore last three emelemts of series
    report['ColumnTotal'] = report.drop(columns=['ColumnMax', 'ColumnMin']).sum(axis=1)

    #cleanup
    report.index = report.index.droplevel(0)

    report.rename(columns = {values_matrix: '',}, inplace = True)
    report = report.apply(pd.to_numeric) # convert all columns of DataFrame, but it re-introduces nan, so replace nan again below
    report = report.round(3)

    #report = report.iloc[:, :-4] #remove unwanted columns
    report.drop(['ColumnMax', 'ColumnMin'], axis=1, level=0, inplace=True) #dropping  colum from multi-index columns

    return report

In the above snippet, many of the variables may represent any of the container data structures. It would be useful to go through and assign an appropriate data structure as an exercise. I am sure you can see that the above code snippets are full of data structures.It is just data structures. If you are not proficient in handling data structures, then you may not comprehend the logic.

HTTP Session and Cookies

A little about Session and Cookies

In this session (no pun intended), I will try to discuss the topic of HTTP sessions and cookies. Prior understanding of HTTP is assumed, but just to refresh your mind, HTTP is a protocol governing communication between a client and server using requests and responses. A client sends a request, the server receives the request, processes the request, then sends back the response to the client, and finally the client receives the response. That is the HTTP request/response cycle.. 

Although this post will show examples using Python (or specifically Flask microframework), the theory should generally apply to any development language/framework.

We start by understanding Session and Cookies. Session and cookies are used by different websites for sharing user's data across different pages of the site. They both track information specific to a given user of a website. A session is maintained through a global variable stored on the server side, while cookies are maintained through small files (typically maximum 4KB, but may depend on the browser) stored on the browser or client side of a web application.

By design, HTTP requests are stateless. It means that a series of HTTP requests from the same browser (client/user) appear to the server as totally independent. The server cannot tell that they are all coming from the same browser or user. Typically, web servers don't maintain any information from request to request. A web-developer, therefore, would find it difficult to bundle together a series of requests from the same client. What if each request could have an identifier which is specific to a client? The answer to this question can help us to solve this problem using sessions and cookies.

More about Session

A session is used to save information on the server temporarily so that it may be used across various pages of the web application.  A session starts when the user logs in to a website, and ends when the user logs out or closes the browser. Without session, the data available on one page would not be available when the user opens another page. With session, the data is made available using the session variable, which is a global variable. Session allows you to store information specific to a user from one request to the next.

In Python, we can set a session variable (global) as a dictionary like these examples

session[‘username’] = ‘rorama’
session[‘username’] = request.form[‘username’] # read from a form request

Throughout the above session, we can access username from anywhere in the application using this

username = session.get(‘username’)

As you can see above, one must be familiar with data structures (dictionaries specifically) to be able to effectively use sessions.

It is secure to use sessions since they are stored in binary or encrypted form on the server. Each session is unique for each user (and is identified with sessionID), and an application can have unlimited number of sessions.

A simple use case of session is where we want each application user to have a custom value for number of rows that a page can display after they login. After login, we can have the user set the value for rowsPerPage in a form, and then we store it in a session variable as follows

session['rows_per_page'] = int(request.form[‘rowsPerPage’])

Then we use the following throughout the application to show pages with the defined number of rows

session.get('rows_per_page')

More about Cookies

Cookies are text files with small pieces of data, e.g. a username and password, that are used to identify your computer as you use a computer network. As already noted above, cookies are stored on the client side of a web application. When a user first visits a website, and if the site is configured to use cookies, then the server will send data to the user’s computer in the form of a cookie. All subsequent requests by the client to the server then transmit along cookie information (sessionId) in the header.

Cookie data is labeled with an ID unique to you and your computer - your Session ID. When the cookie is exchanged between your computer and the server, the server reads the ID and knows what information to specifically serve to you.

The aim of Cookies is to remember and track data that is relevant to customer usage for better visitor experience and website statistics.

A cookie expires on the lifetime set by the user.

The information stored in cookies is not safe since it is kept on the client-side in a text format that anybody could easily access, but cookies can be disabled if necessary.

In Python, to access cookies you can use the cookies attribute of the request object. To set cookies you can use the set_cookie method of response objects. The cookies attribute of the request object is a dictionary with all the cookies received from the client.

Reading cookies

from flask import request
  @app.route('/')
  def index(): 
      username = request.cookies.get('username')

Storing cookies

from flask import make_response
@app.route('/')
def index():
  resp = make_response(render_template(...)) 
  resp.set_cookie('username','rorama') 
  return resp 

For dictionaries, note that we use request.cookies.get('username') rather than request.cookies['username'] so that the code doesn't break in case request.cookies is not set.

That is it for today.

My Journey with the Migration from JQuery to Vanilla Javascript

As a front-end Javascript library, JQuery is so easy and versatile to use, the reason I have used it all these years for HTML document traversal and manipulation, event handling, animation, Ajax, etc. But even the best dancer leaves the stage at some point. It is time to say bye to JQuery, albeit in a painful way. Life must go on without JQuery. 

A few months ago, I embarked on a tough journey – a journey which I knew would take me long. I am still trekking through the jungles, mountains and wilderness. At the end of the journey, I must have migrated all my frontend JQuery code to Vanilla (pure) Javascript, or some other modern Javascript library such as React. This appears to be the trend among the developer community given the developments in the Javascript space in recent times, so I will not be left behind. 

The advantage I have is that all the JQuery functionalities will still continue to run normally, so it isn’t like I must complete the migration process now. My systems will not break today or tomorrow because of JQuery. I have enough time for the task. 

In this post, I share some of the migration code snippets that I believe may help others. 

I created two global constants, dqs and dqsa, as short names (alias) for document.querySelector and document.querySelectorAll respectively. This makes it easier to make the function calls later for DOM Manipulation, Event Handling, Ajax, etc.

const dqs = (query) => document.querySelector(query);
const dqsa = (query) => document.querySelectorAll(query);

Provided you are well versed with document methods (which includes querySelector and querySelectorAll), and the CSS selectors (query), it would then be easier to migrate the scripts as I show in the rest of this post. I am going to show as many examples (from my working system) as I can by continuously updating the snippet list below.

DOM Manipulation

  //JQuery
  $("#lastActive").text(lastActive);
  $("#idleTime").text(idleTime);
 
  //Vanilla
  dqs("#lastActive").textContent = lastActive;
  dqs("#idleTime").textContent = idleTime;    

DOM Manipulation

  //JQuery
  $("#flashDiv2Message").html(pendingContactRequests + ' - ' + pendingRequisitions);
  $("#numContactRequestsPending").text(numberContactRequestsPending);
  $("#numRequisitionsPending").text(numberRequisitionsPending);
  
  //Vanilla
  dqs("#flashDiv2Message").innerHTML = pendingContactRequests + ' - ' + pendingRequisitions;
  dqs("#numContactRequestsPending").textContent = numberContactRequestsPending;
  dqs("#numRequisitionsPending").textContent = numberRequisitionsPending;  

DOM Manipulation

  //JQuery
  $(".img-zoom-result").css({"display": "block"});
  imageZoom($(this).attr("id"), "imageZoomDiv");
  
  //Vanilla
  dqsa(".img-zoom-result").forEach(el => el.style.display = "block");
  imageZoom(this.id, "imageZoomDiv"); //zoom this image on imageZoomDiv  

Migrating $(document).ready()

  
  //JQuery
  $(document).ready(function() {  
      //do something
  });
  
  //Vanilla
  document.addEventListener("DOMContentLoaded", (event) => {
      //do something
  });    

DOM Manipulation and Event Handling

  //JQuery
  $('#recordImage1, #recordImage2, #recordImage3, #recordImage4, #uploadedImage').mouseleave(function() {
      $(".img-zoom-result").css("display", "none");
  });  
  
  //Vanilla  
  dqsa("#recordImage1, #recordImage2, #recordImage3, #recordImage4, #uploadedImage").forEach(el => {
	el.onmouseleave = function(e) {
  		dqsa(".img-zoom-result").forEach(el => el.style.display = "none");
  	}
  });   

Migrating Ajax to Fetch API
Note: Please do not think that I am still using JQuery in the Vanilla version below where you see fetch(`/get_district_by_id?id=${elemId}&whichAction=${whichAction}`.
Rather, I am using backticks -  template literals - for expression interpolation  such as `${varName}`.

  //JQuery
  $.ajax({
      url: '/get_district_by_id',
      data: {
          id: elemId,
          whichAction: whichAction
      },
      dataType: "text",
      type: 'GET',
      success: function(res) {
          try {
              //do something
              }
          }
          catch {alert('Error loading data');} //string is not JSON object
      },
      error: function(xhr, status, error){
          var errorMessage = xhr.status + ': ' + xhr.statusText;
          alert('Error - ' + errorMessage);
      }
  });  
  
  //Vanilla
  fetch(`/get_district_by_id?id=${elemId}&whichAction=${whichAction}`, {
      method: 'GET', // or 'PUT'
      headers: {
          'Content-Type': 'application/json',
          'X-Source-From-Fetch': 'True',
      },
  })
  .then(response => response.json()) //de-serialize/parse json to javascript object, can now access using response["key"] or response.key
  .then(data => {
      try {
          //do something
      } catch (error) {
          alert(`File: ${error.fileName} \n Line No: ${error.lineNumber} \n Message: ${error.message}`);
      } //string is not JSON object
  })
  .catch((error) => {
      console.error('Error:', error);
  });    

Trigger Click

  //JQuery
  $('#sendbutton').trigger('click');

  //Vanilla
  dqs("#sendbutton").click();  

Trigger Mouseenter
Note: mouseenter doesnt work like click() trigger for good reasons

  
  //JQuery
  $("#loggedInUsers").trigger("mouseenter");

  //Vanilla
  dqs("#loggedInUsers").dispatchEvent(new MouseEvent('mouseover', { 'view': window, 'bubbles': true, 'cancelable': true }));

Hide Spinner

  //JQuery
  $('#spinner').removeClass('fa-spin').parent().css("z-index", -9999999999).hide(0); 

  //Vanilla
  dqs("#spinner").classList.remove("fa-spin");
  dqs("#spinner").parentElement.style.zIndex = "-9999999999";
  dqs("#spinner").parentElement.style.display = "none";

Closest element

  //JQuery 
  $(this).closest(".modal");

  //Vanilla
  this.closest(".modal");

To-do

  
  //JQuery
  
  //Vanilla  

To-do

  //JQuery
  
  //Vanilla  

To-do

  //JQuery
  
  //Vanilla  

Attention to Detail - PEP8 (Python) and Moving Round a Parked Car

When Guido van Rossum – the creator of Python – wrote PEP8, he had a visionary idea. Back to PEP8 later.

Ordinarily, many people would not see any difference between the following lines of ordinary English sentences:

I want to become a programmer.

I want  to become a programmer .

The second line has two spaces between “want” and “to”, yet the first line has only one space. The second line also has a space just before the ending period (dot), yet the first line does not. To a programmer, these are serious differences which should not be ignored. As an interviewer for a programming job, I would want to test if a candidate can easily spot such differences. If one has a difficulty seeing the differences between the two lines above, then they would likely have difficulty adhering to programming rules. As I always say, programming rules are not so forgiving.

Let me give another example. If, in one instance in a report, you write "PEP8 (Python)". In another instance in the same report you write "PEP8(Python)". As long as you are aware of the two instances, but you have made no effort to distinguish between them, then clearly there is a problem with attention to detail.

This brings me back to PEP8.

PEP8 (Python Enhancement Proposal) is a document that provides guidelines and best practices on how to write Python code. It was written in 2001 by Guido van Rossum to improve the readability and consistency of Python code. This document is one good demonstration of  why recruiters are always interested in a candidate's level of attention to detail. You cannot do so much with PEP8 if you do not pay attention to detail.

Consider the following function:

def function1(c):

	""" this function does a computation by accepting one parameter, and
	returns the computed value
	"""

	a = 1
	b = 2
    
	return c * (a + b)

Now, without observance of PEP8, you could easily write a=1 instead of a = 1, which is not good programming.

When I implemented my first Python project, I had not yet heard about PEP8, yet it had been in existence for several years. I can only blame lack of time for that. When I deployed the system, all was well, until a reviewer pointed out the lack of adherence to some of the PEP8 provisions. It was not too late. I had to do a lot of code refactoring to ensure as much compliance with PEP8 as possible. The beauty is that it enhances and standardizes readability and maintenance, and promotes best practice.

Finally, a practical example of attention to detail applies to my daily life – Moving Round a Parked Car before driving off. 

People always wonder why I first move round my parked car before I enter to drive off. For me, it is just another unwritten rule to do this. So I get questions from onlookers such as: is there a problem with your car? I say, No, I just move round to make sure that everything is okay before I drive off. Since I hardly see anybody else doing that, sometimes I wonder what is wrong with me. But it is already a rule for me, so I have to obey it.

How would it feel to drive a car off with a flat tire, or to drive over a cat that is sleeping just next to your tire? Not even just a cat. In a true story, someone reversed a parked car a few years ago and killed a baby who was seated just next to the tire. The mother of the baby had just gone to pick something and left the baby there. The things we do in our everyday lives define what we can or cannot do. Any lesson to learn, especially for the techies?

 

Demystifying args and kwargs in Python

If you use Python for your data science or software development projects (or for amything else), then you have most likely used or seen args and kwargs. They may look intimidating at first, but they are easy to understand and use. 

To understand args and kwargs, let us start by defining a simple function

def print_str(str1):
	print(str1)

The above function takes one “required” parameter called str1, hence you can call

print_str(“my string”)

You cannot call

print_str()

because one required argument must be passed in the function call. The above example is a trivia.

What if you wanted to pass another one or two or more arguments which are optional. Then you can use args and/or kwargs. Both provide a handy and flexible way to handle optional and variable number parameters in functions. They have many practical use cases. 

Please note the difference between parameters and arguments: parameters are used in function definitions, while arguments are used in function calls. Sometimes, parameters are referred to as Formal arguments, while arguments are referred to as Actual arguments, but don't allow this to confuse you.

In the above example, def print_str(str1) means that str1 is a parameter since it is being used in a function definition, while print_str(“my string”) means that “my string” is an argument since it is being used in a function call. I hope that clarifies a common confusion. Now let's move on to args and kwargs.

args (stands for arguments)

In the following example, a corresponding function call must supply one required argument (str1) and a variable number of positional arguments (args).

def print_str(str1, *args):
	print(str1)

	for s in args:
    	print(s)

 

Note the following:

1. args must have the unpacking operator ‘*’ before it.
2. args could be named anything, provided ‘*’ appears before it. We just prefer to name it args by convention.
3. args is a collector of optional positional arguments. It collects the arguments and packs them into a tuple, which is an iterable.
4. args works for positional arguments.
   

Now, you can call the function by passing as many “args” as you want. Note that args are also positional arguments, just like the required arguments.

        print_str(‘my string’, ‘arg1’ , ‘arg2’, ‘arg3’, ‘arg4’)

        print_str(‘my string’, ‘arg1’ , ‘arg2’)

 

kwargs (stands for keyword arguments)

In the following example, a corresponding function call must supply one required argument (str1), a variable number of positional arguments (args), and a variable number of keyword arguments (kwargs).

def print_str(str1, *args, **kwargs):
	print(str1)
    
	for s in args:
    	print(s)

	for k,v in kwargs.items():
    	print(k,v)

Note the following:

1. kwargs must have the unpacking operator ‘**’ before it.
2. kwargs could be named anything, provided ‘**’ appears before it. We just prefer to name it kwargs by convention.
3. kwargs is a collector of keyword arguments.  It collects the arguments and packs them into a dictionary, which is an iterable.
4. kwargs works for named arguments.
   

Now, you can call the function by passing as many “kwargs” as you want. Note that kwargs are NOT positional arguments.

        print_str(‘my string’, key1=‘kwargs1’ , key2=‘kwargs2’, key3=‘kwargs3’)

        print_str(‘my string’, key1=‘kwargs1’)

Before I sign off, it is important to note that you must have good understanding of functions, data structures (specifically tuples and dictionaries) in order to effectively use args and kwargs.

I hope that makes it easier now. Your feedback is welcome.

The Art of Problem Solving and Critical Thinking (a Computer Science skill)

The Art of Problem Solving and Critical Thinking. I like this topic with passion ... after all it is the foundation upon which Computer Science is built.

Recently, a friend approached me with a problem. It goes thus:

"We have a telephone system in our office, serving about 200 staff - literally translating to 200 telephone extensions. The system is VOIP-based, and it uses some opensource PBX system. Many of our staff are sort of indisciplined when it comes to phone usage, and they can make many unnecessary telephone calls, resulting into unmanageable costs. We are unable to limit calls made by staff. Our IT team has tried to research around but the opensource PBX system does not have functionality that can limit calls. The team says it is impossible to limit calls. Can you please help us think out a solution that will result in cost containment?"

I said Wow! This is the sort of practical problem that requires critical thinking. There is no problem that has no solution. For starters, one needs to have the freedom to think of possible solutions outside the existing system (thinking outside the box).

I found the above problem too obvious to resolve because I was faced with exactly the same problem years ago, only that in that case the client was using hardware PBX system, which made it even more challenging. Nonetheless, I solved that problem using first principles without much ado. It was just another piece of cake. The solution did not only result into significant cost reduction, but several other benefits to the organization (that can be a story for another day, if you ask me).

Without a doubt, it is all too obvious that this is a problem which requires a logical IT solution. An important component of the solution is to appreciate that with the opensource PBX system (software, rather than hardware), the following would hold:

1. Each staff has a user telephone account in the database
   
2. Each call made is saved in the database

With the above background it is not hard to see that one can easily implement the following additional functionalities:

1. Create at a minimum the following additional database fields in the table containing user accounts
• quota or call_limit (int)
  
• call_amount (int)
  
• active_status (boolean)
  
2. Every time a user makes a call, update the call_amount above by adding the amount of that particular call.
3. Implement a scheduled Job that runs every so often (e.g. five minutes). This Job checks if the field call_amount has exceeded the field call_limit. If so, then that user account would be deactivated by changing the field "active_status" to False. By doing so, the user will not be able to make calls.
   

Simple solution above, isn't it? But that is just a bird's eye view of the solution. To implement this, you will have to dig deeper. The ideal person for such is a full-stack developer because you will need knowledge of the following as a minimum:

1. Relational Database Management System (with good SQL knowledge)
2. Frontend (Javascript, HTML, CSS)
3. Backend (any backend language - but I would personally use my favorite Python of Java)

I have heard many talk about coding as a "skill". True, you need to know how to code in order to implement solutions, but do not forget that finding the solution could be even more challenging and time consuming than coding. And coding can be done in any of the well-tested programming languages or database management systems or platforms as long as you are proficient in them.

Finally, when I told my friend that I would share this knowledge on my blog, he was like: No, No, No, aren't you sharing some sort of intellectual property? Which made me burst into laughter, for the following reasons among others:

1. This particular problem and solution should be a no-brainer even for the not-so-accomplished programmer.
   
2. Most of the things I have learnt over time were because other people shared them. I personally get most of free knowledge off the web. So why not give back something if I can?
3. Even if I share the above solution, it is only the tip of the iceberg. There are too many problems out there that require analytical thinking. Every problem is solved on a case-by-case basis. For anyone to use such a solution, they still require to be able to further analyze and customize the solution to their needs. The gist of the matter is that you will not simply teach someone analytical or critical thinking skills - at least not overnight.
   

So when I hear people talk about any manner of software or IT solutions as Intellectual Property, sometimes I do not understand what they mean. Sometimes I believe that it is because not everyone appreciates what software is made of.

Back to English Grammar and Computer Programming

In an earlier post, I had argued why/how English grammar is so similar to computer programming syntax. I noted, however, that computer programming syntax is not as forgiving as the English grammar. Make a mistake in English, people may understand (or misunderstand) you, and you will walk away with it. Make a mistake in programming syntax, the computer will say NO, you are wrong, I cannot understand you, so I will not execute your instructions!

Today, I want to share two other common mistakes in English grammar (last time I talked about misuse of commas). Look at the following:

1. Me and my friends went swimming
2. Please join my son and I

I hear and read the above sort of mistakes every day … and unfortunately, those who hear or read such things every day will firmly believe in them, which is very dangerous. This is even worse for children since the mistakes will stick in their heads forever ... just like I cannot believe up to now that there has never been a World War III simply because I heard something from my teacher when I was a kid, and it has remained glued in my head to-date.

To appreciate the injustice done to the Queen’s language with the above two examples, one simply needs to understand the difference between subject and object in English grammar.

A subject commonly does something (i.e. comes before a verb), especially in a declarative sentence, where the subject appears before the verb ("The man shouts") – here “man” is the subject. In an interrogative sentence, the subject usually follows the first part of a verb ("Does the man shout?")

An object is a noun, a noun phrase, or a pronoun that is affected by the action of a verb. In other words, an object is governed by a verb or a preposition. Something is done on an object. An object comes after a verb.

Once you are at home with the concept of subject and object, then you can hardly make mistakes such as in the above examples.

Just to get a better grasp, consider the above examples to be similar to the following:

1. <Me and my friends> went swimming, or generically <subject> went swimming
   
2. Please join <my son and I>, or generically Please join <object>
   

In number 1 above, <Me and my friends> is a subject; <my son and I> is an object. So ask yourself: would you ever say “Me went swimming” or “Please join I”? Would you ever say "Me go to school" or "Please tell I the truth"? Obviously not, and therefore, you should never make the above mistakes. The correct forms are:

1. My friends and I went swimming
2. Please join my son and me

As a general rule, use “I” when the pronoun is the subject of the sentence, and use “Me” when the pronoun is the object.

I cannot over-emphasize the similarities between the English grammar and computer programming. I just like the discipline that computer programming inculcates in a programmer. It’s like the kind of discipline we hear exists in the military.

Next time, I want to compare and contrast “statements” in English and computer programming (or logical math for that matter). That’s another very interesting topic for computer programmers.