As software developers, the one unwritten cause for misery is that NOT many have the ample
time they deserve to do what they want to do. Moreover, software development involves a lot of reading, researching and documentation in addition to programming or coding as many seem to know.
Lack of time sometimes leads to
not-so-quality products because a developer has not put their best foot forward
in the implementation. Time is one of the greatest assets. We need enough time to understand
what we are doing. Doing things in a rush-rush can also result into
spaghetti-code because clean code is not a top priority when the boss is
standing right over your shoulders waiting for you to deliver an urgent
functionality. To the boss, the priority is just about functionality - which is understandable. To the developer, the priority should be both functionality and code quality - including the sometimes-overlooked documentation.
I have come up with my latest checklist of the items for a web-based
database development project. This list will be brushed up soon to make a
useful reference for other developers. These are the things I found useful to
include in the system. Even if the customer has not requested for a functionality, I use my experience to determine the usefulness of such a functionality for future customer needs. Therefore, when I get some time, I implement them for potential future use.
Specifically, the latest system I developed is using Flask (a Python micro-framework that gives you the liberty to use what libraries you want - in contrast with Django that comes with most of its standard libraries). Flask is micro-framework, not because it is a micro, but just because it gives you the fundamentals and leaves you with the liberty to choose what else you want. That is a great strength indeed.
The checklist is not just about Flask/Python, but also other backends and even frontends. Moreover, the list should be applicable to any other development language / platform.
In my opinion, the checklist contains many of the standard things that a typical database application would require. At the same time, since developers learn new things everyday (I believe), this list would likely change with time. We are always better that what we were yesterday.
As we read through this, one’s strength in the language fundamentals, data structures, algorithms, system security, best-practice, etc cannot be over-emphasized. I understand that many self-made developers learn by straight away implementing a development framework, and once they can save and display records from the database, that celebration marks the end of further learning. That collection of “developers” miss out on the language fundamentals and best practices; and their products can only be used in that context.
Here comes the good-to-know list (not exhaustive by any stretch of imagination):
1. Database Abstraction / Object Relational Mapper (SQlAlchemy in the case of Flask). Some important aspects are relationships (one to one, one to many, many to many, cascades), model class definition, formulas (computed properties), enums, event listeners (before_insert, before_update, after_delete, after_insert, etc)
2. Templating Engine (Jinja Template in the case of Flask). Jinja is a very powerful templating engine, not very different from the Django template.
3. Reporting and Data Analysis (Pandas, GeoPandas, Plotly, Matplotlib). The urge to have an “inline reporting tool” for charts, lists, summaries, maps, etc is obvious. It provides a quick reference as opposed to a separate reporting framework. Pandas and its associates can done wonders in both analytics and reporting of data.
4. UI (bootstrap) – this speaks for itself. Master how the frontend relates to the backend through requests and responses. Master checkbox, radio button, file upload/download, child forms/tables, modals, navigation, event calendaring, bootstrap inline form, stacking vs grouping checkboxes and radios together i.e. form-check form-check-inline, proper input validation.
5. Language Fundamentals, Data structures (at least thoroughly understand primitive/container data types/structures, i.e. integer, float, string, boolean, list, tuple, set, dictionary, queue (FIFO), stack (LIFO)), Algorithms (at least understand implications of time and space complexity, Big O Notation, etc). These are the things you use every singe second when you are developing an application. You cannot run away from them.
6. Security: Use Flask libraries for proven security - Flask-Login, SQLAlchemy, Jinja all enforce security with their internals. Understand security risks: injections (cross site scripting SQL injection, Cross Sire Request Forgery). Understand HTTP Request Methods, and which one to use in what circumstance. Understand sessions and cookies.
7. Frontend technologies (Javascript, HTML, CSS)
8. Task queuing/scheduling - task queuing. I used Celery.
9. Structuring / re-structuring the application
10. Refactoring
(for clean, efficient, scalable, easy-to-maintain code). Check the quality of your Python code for PEP8 compliance (online tools available - PythonChecker.com, pep8online.om). PEP8 (Python Enhancement Proposal) is about good coding style.
11. Compiling application (Cython)
12. Functionalities such as emailing, database backup, HTML to pdf converter, Chat box functionality – useful for user-collaboration
13. Which
IDE to use to simplify development? The choice is very important.
14. Version control and collaboration (Git and GitHub)
15. On-premise deployment (WSGI)
16. Cloud deployment (test using PythonAnywhere)
17. Learn the debugger and save time
18. Manipulate third-part applications, e.g. Excel - using Excel writer (OpenPyxl)
19. Forms validation - using Flask WTF
20. Audit trail (audit logs) to capture data entry activities
Sample data management and visualization screens are below. Also see sample reports in this post.
Sample Data Listing:
Sample Data Entry Form
Sample Visualization
