Sections in this guide

• Prologue
• I. Introduction
  • The Software Process
  • Question to Consider
  • How much of this article do I need to read?

• II. The basics before you start
  • How can we judge the success of a software project?
  • Why do software projects fail?
  • People on a software project

• III. The Process
  • What are the different phases of a software project?
  • Waterfall approach
  • Iterative processes
  • Do I need all of these phases? 

• IV. First things first: Know what you want
  • What do requirements look like?
  • How much time should I spend on requirements gathering
  • What makes a good requirement?
  • How technical do my requirements need to be?
  • Technical requirements

• V. Design it
  • Divide and conquer
  • From nothing to screens in three days
  • Accelerated design sessions
  • Wireframes and screen design documents
  • Use case documents
  • System flow
  • Where do you stand?
  • Full circle: back to requirements again
  • Technical Architecture

• VI. Implementation Phase
  • How is the implementation phase divided?
    1. General architecture design
    2. Technical screen design
    3. Coding stage
    4. Unit testing
  • Are all the modules implemented at the same time?
  • Before you implement, what is the right technology?
    1. Open source vs. commercial software, overview
    2. Open source vs. commercial software, quality
    3. Open source vs. commercial software, support
  • The different types of programming languages
    1. PHP
    2. Java based solutions
    3. Microsoft .Net
    4. Ruby
  • The right technology: In the final analysis
  • Outsourcing my project
  • Challenges in working with Indian companies

• VII. QA phase
  • Levels of testing
  • Functional verification
  • Performance testing
  • Usability testing
  • Requirements traceability and testing
  • User acceptance testing
    

• VIII. General thoughts around software projects
  • Pitfalls of software projects:Process for the sake of the process
  • Pitfalls of software projects: the anti process
  • Web Application vs. Web site

Prologue

INVESP is a Houston based business services firm that provides a multitude of business offerings including technical writing services. INVESP is also a community resource for those in the administrative and content management fields. You can learn more about INVESP here. We provide a great variety of free information via our blog as well as the article section of our website.

This article is offered as a resource to help individuals, organizations, and companies inexperienced with software development to learn the basics of managing software projects. It is our goal to help you to more effectively plan, manage and complete your software projects on time and within budget. We share this knowledge to help businesses, government, educational, and non-profit organizations successfully complete their software projects from requirements to release.

If you are new to software projects, have read through this document, and require assistance for your project, you may learn more about us here.

I. Introduction

This guide is designed to cover all areas of software projects - from the initial phase of requirements gathering to actually using and releasing the final product. It includes the details of selecting technical resources, steps in completing the actual work, as well as choosing the right technology for your project, and managing the implementation and quality assurances phases of your project.

While there are many academic courses that discuss some of the aspects covered in this article, most of them target those who actually work in the field of software development. Many computer science departments in universities around the country teach a course on software engineering that delves deep into most of the aspects covered here. These courses are not practical for those who simply need a quick reference to help them complete their software projects.

The Software Process

The question is, how important is it for me to have a good understanding of the software development process? Well, the good news is that you have finally decided to complete a software application that you need for your business. The bad news is that if you are not careful the experience can be painful. Making this decision is only the first step on a long journey until your project is completed and you are actually able to use the software. This journey can either be easy, enjoyable and rewarding or it can be painful and costly. The reality is that most software projects end up failing. A study from the early 1990s stated that only about 30% of software projects are actually completed on time and within budget. Think about it. 7 out of 10 software projects will fail, run over budget or take longer to finish than initially estimated.

The purpose of this article is to help you on your journey from the initial steps of choosing the correct technology for your project, through your design phase, to implementation and testing. By having a good understanding of the process and following some of the advice you find in this article, you will be able to minimize your risks and frustrations. Understanding the different stages of software development is essential in completing your project successfully. With careful planning and excellent execution you will be able to deliver your project on time and within budget.

Question to Consider

How do you go about starting your project? What are the different phases your project will go through? What different technologies should you use for your project? What company should you hire? The list of questions goes on. One of the only answers you are sure of is that you do not want to hire a full time software developer to complete the work. This leads you to the option of outsourcing, which will be explained in further detail later on.

How much of this article do I need to read?

If you are serious about completing your software project on time and within budget, I recommend you read this guide from start-to-finish. There is a printable MS Word version for those who prefer, as well as dozens of linked-to resources on other sites and pages that are worthy of your attention. Although this guide is long, I've attempted to remain precise and concise.

II. The Basics before you start

How can we judge the success of a software project?

Software engineers and managers debate which factors should be used to determine the success of a software project. Below are the factors we use to determine our success on different projects we work on.

• The software application must fulfill all the business requirements it was designed to meet: This is essential to the success of the project. Missing or incorrect functionality does not allow for the successful completion of a project.

• The software application must be designed well: This is usually unapparent to system users. However, poor design can cause an application to perform slowly or result in maintenance problems.

• The software application must be implemented well: A well designed application must have well written code in it. Poor code will show its ugly face with poor performance and maintenance nightmares. Imagine paying a ton of money for an application. You get the application and are happy with it. After using the application for a couple of months, you no longer see the need to pay the company who wrote the software for maintenance since everything looks great. A few months later however, you discover a bug. It is a small bug and you estimate it should not take more than a couple of hours to fix. But when you get the estimate to fix the bug you are shocked by the number. The code is so badly written that it will take days or even weeks to fix the small bug.

• The software application must be completed on time and within budget:  Software projects are inherently complex and require careful planning.  Proper planning ensures that the project does not move away from its targeted goals and also ensures that each phase or deliverable of the project is completed on time and within budget. A phased delivery reduces risk of estimation errors while better allowing you to keep tabs on growth. Project management is the key to completing your software application on time and within budget.

You can come up with your own list of factors to judge the success of your software project. What is important is that you go through that exercise before you actually start the project. It is also not necessary to assign the same weight to all of the factors. Some applications are great with only 90% of the requirements satisfied as long as the project is delivered on time and within budget. On other projects, such as software controlling life support devices, meeting 100% of the requirements is critical even if that means delaying the actual completion date of the project for months.

Why do software projects fail?

Software projects fail because of many reasons. Some of the most common reasons I have seen include:

• People! Your project is destined to fail if you have either the wrong engineering team or the wrong management team on it. Team members need to work closely together to accomplish the final goal of delivering on time.

• Unrealistic timelines: Countless times deadlines are unrealistically set by managers who simply pick a date with little or no regard to how long it actually takes to accomplish the tasks.  The end result? Either the deadlines are missed or they are met but with nonfunctional software. Estimating the time correctly for a software project is a topic for an entirely different blog.

• Wrong technology: Is java the best technology for your project or is it .Net? Maybe you should go with PHP or ruby. What database should you use? Access Mysql, or maybe Oracle. Can you even afford Oracle? There are an endless number of technologies out there as well as an endless number of people who will tell you that each of these technologies is the right one for your project. Choosing the correct technology that will deliver the best results for you takes both a good understanding of your own software requirements and a good understanding of each of the different technologies available.

• Poor design or poor implementation: Imagine spending thousands of dollars and hundreds of hours on an application. The application is finally ready to be used and everyone is excited. Once the application is turned on it takes users about 10 minutes to login to the system. Or how about cases where the application performs really well when you test it but when you have 100 users on the system, it starts crawling or even comes to a screeching halt. It happens more than software companies would like to think. A great application needs to have both a good design and well-written code.

• Unrealistic budget: I have seen companies that take the cost estimate for a software project and cut it in half without eliminating any of the requirements of the project. In plain English, that does not work. You will end up with a disaster. If you have to cut costs, make sure to cut some of your requirements as well.

People on a software project

Software projects bring together people from different business areas who are all trying to achieve one goal. Think about your own project. There are two entities involved; the entity which initiated the project and the entity doing the technical work. Within these two entities you probably have the following players:

• The entity which initiated the project. That is probably you or your company. Most likely your technical knowledge is insufficient to design
and build an application that will satisfy your requirements; however you
have knowledge and experience with the business processes and know what is
required. Your goal is to implement an application that completes the task
you need in a timely manner and within budget.

• The entity doing the actual technical work will contain the following roles:

o A software architect - an individual who leads the technical staff on the project. Just like a building architect, a software architect is responsible for understanding the business requirements and designing an application that meets those needs. He ensures that technical standards are maintained and adhered to in your application and oversees the engineers who will construct the application based on the design specifications.

o A Project manager - the person responsible for planning and development of all project deliverables. The project manager is responsible for managing the budget and work plan and all project management procedures including scope, risk and issues management. Usually the person you interact with the most, the project manager is responsible for the overall success of the project.

o The project team – a group of people consisting of technical resources assigned to work on the project deliverables. The project team is responsible for completing assigned work within budget, time and quality expectations. The technical resources may include internal staff as well as external resources. The technical staff - individuals who are well-versed in technology, at least you hope they are, but they lack a good understanding of the business processes.

Communication with your architect is extremely important, as he is the mind behind your application. Keeping the focus on meeting the business requirements is critical to designing a solution that meets the needs of your business. On smaller projects, technical staff, project managers and architects are roles that can be assumed by one person.

III. The Process

What are the different phases of a software project?

Traditionally, a software project goes through the following phases:

• Requirement gathering phase: Compile a list of what you want your software application to perform. At the end of this phase, you should have a list of requirements or specifications for your application. Donn Le Vie wrote an excellent article that discusses issues around requirement gathering phase.

• Design phase: Take the requirements you developed in the previous phase and use them to start designing the actual application. By the end of this phase you should have design documents for your application.

• Implementation/development phase: This is the stage where the software developers you hire to do the work takes your design documents and starts writing code to “implement” the application. If things are done correctly up to this point, at the end of this phase you will have a working version of your application.

• Quality assurance/testing phase: I have yet to see an application that is fully ready to use after the end of the implementation phase. Software and bugs go hand in hand. This phase tests the application to make sure it is free of errors and that it operates correctly.
• Release phase: Application is released to the users.

• Maintenance phase: This phase is ignored in many projects since it comes after the completion of the project. It covers any changes to the application after the application is completed and the actual users start using it. It can include fixing bugs or adding new screens or functionality to the applications. Most software companies that complete the project for you will offer some sort of maintenance or warranty on the application. However, there is always a charge for that service.

Some of these phases can be broken down into smaller phases. For our purposes, the division outlined above is acceptable. It is important to keep in mind that in many cases the separation between the phases is not as distinct as some people might think. It depends on many factors including the size of the project, the culture of your own company, and the formality of the software development process in your firm. There are two main approaches to the software process.

Waterfall approach

The waterfall model is the oldest and best-known process where developers follow these steps in order. In traditional software development, each phase is assigned a certain time limit and the project moves forward from one phase to the next.

Iterative processes

New approaches to software development take a more iterative approach. You will continue to have the same outlined phases above however these phases are carried out in extremely small steps. In this approach coding or implementation of modules does not have to cover 100% of the functionality prior to a user seeing the actual screens. By allowing users to see screens and modules early on, the client who does not know how to define what they want will be able to make adjustments to the system right away.

Instead of waiting for all implementation to be completed before testing the application, you can choose to start testing the application modules as they are implemented.

Do I need all of these phases?

YES. I can not emphasize enough the importance of understanding the phases involved in the software process. It is equally important to understand that the process can be as straightforward or as complicated as you make it. For instance, if all you need is to add a new screen to a pre-existing system then the whole process might take less than a day to complete. On the other hand, if you are starting a new project to design and implement an application you will use for next few years, it makes a lot of sense to invest considerable amounts of time in each of the phases above.

Therefore the time invested in each phase varies vastly depending upon the specific details of the project. I worked with a client whom I will refer to as company X. A director decided they needed to be online. He called for a meeting where the whole website for the company was designed in less than an hour. My client did not give careful thought to the purpose of the site. They did not think about the functionality their site needed to perform or serve, nor did they consider the general flow of the site. They hired a graphic designer to complete the work and the site was released within a week. The end result was that the site was not usable and company X lost a lot. If the process we outlined was followed, Company X could have saved themselves valuable time, effort and money.

IV. First things first: Know what you want

The first step towards a successful project is by knowing your requirements. In the technical fields, requirements are generally referred to as SRS or software functional specifications.

I do not want to get into a lengthy discussion around the definition of a requirement. In simple terms, a requirement should state clearly a functionality that you want your software to perform.

Projects vary with the complexity of their requirements. I have worked on projects where the number of requirements the software had to fulfill was close to 2000 requirements. Other projects had less than ten requirements. It all depends on your business needs. The only rule I like to follow when gathering requirements is to be as detailed as you can in specifying requirements. That will save both you and the company you hire a lot of headaches in later stages.

What do requirements look like?

Here are some examples of requirements:

• The system must provide users with the ability to register
• The registration screen must collect a username, address, email, and company
• The system will require a user to submit a user name and password to login to the system.

Requirements are the building blocks or blue print for any project. The more solid and well-defined your requirements are, the more stable your project will be.  If you are planning to hire a company to complete the project for you, software developers will use the requirements to give you an estimate of the length of time it will take them to complete the project.

Picture this: You hire a company to complete your software project and they deliver the system for you. You expect the login screen to allow the user to attempt to login three times. If a user fails to login after three attempts, his/her account should be disabled. The actual system you receive allows the user to attempt to login a countless number of times. You are upset, so you call the software company and explain the issue to them. They tell you that this was not specified in the requirements. Oops! What do you do?

Now the software company has many things they need to consider. They are not at fault. This is an additional functionality that they did not estimate in their original bid. Even if they decide to go ahead and add the functionality, they will have to retest some parts of the system to make sure that this new addition does not break anything else. All of this could have been avoided if the requirements were clear from the start.

How much time should I spend on requirements gathering?

Again this depends on your project. If what you need is a simple modification to a web page, you will probably spend less than half an hour defining your requirements. On the other hand, if you are looking to build a system that you are going to use for the next few years, it might be worthwhile to invest days or even weeks defining requirements. There are companies that specialize in writing requirements.

What makes a good requirement?

Here are some of the characteristics to insure good quality requirements:

• Your requirements must be complete: again try to detail all aspects of the system and the functionality you want to complete.
• Your requirements must be unambiguous: clear requirements make everyone’s life easier. Although in the end requirements are expressed in a language which by definition is ambiguous.
• Your requirements must be testable: You should be able to test the system to see if it fulfills each one of your requirements.

How technical do my requirements need to be?

In general, your requirements should not be technical at all. When you are writing the requirements document, your focus should be on the functionality of your system. The type of application server or database the application is going to use is not an important concern at this point.

What is important is to be realistic when it comes to the requirements. People who include requirements that are simply not possible technically end up disappointed in the end. Oftentimes, business users do not have a good grasp of technical concepts which inhibits them from discerning whether a certain requirement is technically feasible or not. Even if it is possible to accomplish a task from a technical perspective, sometimes the time and effort required do not comply with the specified limits. Welcome to the land of software development. As you go through the different stages you will discover that the sooner you get technical staff involved in your process the better off you are. The right technical person to get involved at this stage is a software architect. A software architect should have enough design experience to help you produce functional requirements that are manageable from a technical perspective.

In an article, How to start a startup, xxx argues that without technical staff involvement in the early stages, a project is doomed to fail. I am not sure I can state that assumption with the same certainty as the author. But I must agree with him that this can be one of the main reasons many companies fail. Developing requirements that are not feasible in terms of the technical aspect is not a good idea in any respect. An example can be seen when looking at many of the dot com companies from the late nineties who were unable to succeed due to unrealistic requirements put in place by inexperienced business users without the help of technical staff.

Technical requirements

Although most of the time, your initial requirements should not be technical, sometimes there may be a need to develop technical requirements for your project early on. This can be confusing; however it does not have to be. Including technical requirements depends on your project. A small project will have very little technical requirements if any. The technical requirements might simply cover the basic important aspects of your system. These include: Performance, security and design.

For example:

• Performance: The system must have a general response time of less than 5 seconds.
• Security: The system must provide the users with secure transactions.
• Design: The system must adhere to general concepts of OOD.
  The purpose of these requirements is to provide a safety net for you if you decide to outsource the project.
  

V. Design it

So you have your requirements, you have a good understanding of what functionality you want the system to provide, now it is time to take these requirements and translate them into screens. Here is the approach I like to follow when designing a system.

Divide and conquer

Take the requirements you developed and divide them into logical functional grouping. How do you do that? Look at the system from the user prospective. Think of the user flow in the system. Let’s look at an e-commerce site. Here are some of the general functions in the system:

1. A login process
2. Navigation through the catalog pages after logging in.
3. The users add items to their cart.
4. The user checks out of the system.

Each one of these steps actually covers a functional area. For instance, as simple as the login process seems, it includes the following:

Login screen: Does the user have an account, does he need to create a new account or did he forget his password? Based on this we just added two more screens to our system:

1. Create a new account screen
2. Retrieve password screen

What if the user attempts to login three of four different times and fails, should the system suspend his account for security reasons? If so, you just added one more screen.

So, our four screens for login create what I call a module. Let’s take the next stage, navigation through the catalog. Below is an example of what the catalog screens might include:

1. There is the header navigation where all the categories are displayed.
2. If you click on a category it will expand to its subcategories.
3. After you drill down, eventually, you arrive at the listing of all the items in a category.
4. Click on an individual item and the system displays the items page.

That is another module. Of course all of this is simply from the user side. It is important to keep in mind that you will most likely have an administration site as well. To better understand that, think of how the items are added to your catalog. You will need an additional screen to add any item to your catalog. So, you will also need different modules to handle that.

From nothing to screens in three days

Designing a system can be a daunting task. Where to start? How to go about designing all the different modules? Allow me to ease you into it. The following approach will help you establish a good starting point. This approach is referred to as the accelerated design approach or accelerated design sessions. I have personally seen this approach used to successfully design a system that had 300 screens in just 3 days. So, unless your system has more than 300 screens, chances are it will work for you as well.

Before we outline the approach, let’s remember that at this point we have a list of the different modules in the system and the different screens in each module. To be able to conduct the accelerated design sessions you will need to get both the business design team and the technical staff involved.

Accelerated design sessions

1. Start with one module in the system. List all the screens in that module.
2. Pick the first screen on the list. On a white board, record all the fields you want the screen to display. This is not necessarily a comprehensive list of all the possible elements but you should capture close to 80% of possible elements on the screen. For example, if we are starting with a login screen, our white board will have: login name, password, forgot password, and register elements listed.
3. After all the elements are listed on the board, try to design the layout of the screen. The design does not have to be fancy. All you need is to capture the general layout. Our login screen will have the login on the top, the password below it. Below them, you will have forgot password, and register.
4. You are done designing a screen! Now capture the list of elements in a word document.
5. Repeat for the rest of the screens in the module.
6. Repeat for the rest of the modules in the system.
   

The truth is that this approach sounds very simplistic and that is the beauty of it. By the end of this exercise you have the general layout of most of the screens in the system.

Wireframes and screen design documents

Having your screens designed on a white board is a great start, but now you need to translate that into html. If you have a design person on your team, he will be able to take the designs you captured and translate them into wireframes of the system. Many companies use Microsoft Visio for that purpose. Another important thing you want to capture is the type of data for each of the elements on the screen. For example, is this field a text field or is it numerical? Is it a date field? You also want to capture the size of the different fields. How many characters do you want the first name field to be? Lastly, you want to capture any validation that must be done on the field.

For example, if you are asking a user to put their email address in a text box, do you want the system to validate that user actually put a real email address or not? Do you want to make a certain field mandatory? For instance, a user will not be able to check out unless they provide you with a credit card number. Sounds complicated? Well, here is a template that you can fill for each screen in the system. Filling a screen template will help explain this step of the process better.

Use case documents

A use case document captures how the user interacts with a screen. There is so much literature around use cases that people sometimes make them more complicated than necessary. A use case simply describes how a user interacts with a screen in the system. For example, one way a user interacts with the login screen is by filling in the correct user name and correct password and then clicking submit. That is the general flow of the system. Another way is for the user to fill in an incorrect username or password. The use case describes what happens in that case. Usually most screens will have a main general flow and one or two exception flows. The use case will outline each possible scenario, how the user gets to it and most importantly how the system will behave in response to each scenario. Filling a use case template will help explain this step of the process better.

System flow

Despite having accomplished an important step by completing the screens, it is still necessary to think about your system flow. What page should the user be taken to after they successfully login to the system? What are the steps of the checkout process? Answering these questions will produce a workflow document. The workflow document can get extremely complex so it is important to proceed with care.

Using the login screen as an example, there are three possible scenarios: User inputs login/password correctly and clicks submit ? next screen will be the catalog screen User inputs login/password incorrectly the first four times ? user remains on the same login page with an error message telling him that he did not put the correct user name/password. User inputs login/password incorrectly for more than four times? User account is suspended. User clicks on the forget password link? User is taken to retrieve password screen.

Your workflow document for the login screen should capture these possible scenarios. Filling a workflow template will help explain this step of the process better.

A question that is heard a lot is “What is the difference between a use case and workflow document?” In many systems there is no need to separate the two. Simply put, use case documents describe the interaction between the user and a single screen in the system. A workflow document, on the other hand, describes the flow between multiple screens and depicts a general process in the system. Some of the details of the workflow document will be captured in a use case document but not all of them. Filling a workflow template will help you better understand this step of the process.

Where do you stand?

By the end of this step, we should have the following:

1. A requirement document
2. Screen documents
3. Use case documents
4. Workflow documents
   

Full circle: back to requirements again

This is the last stage in your business design. It is essential to ensure that the design artifacts you came up with fulfill all the requirements you developed. To do so, take each one of your requirements and write it down in the design document section that fulfills it. For example, one of the requirements we outlined was, “The system will require a user to submit a user name and password to login to the system.” Putting that requirement in the screen documents or the use case document for the login screen is part of a process that is referred to as mapping or traceability. It ensures that the system is designed to meet all the requirements that were developed in the initial phase.

Technical Architecture

Okay, I know you thought we would avoid talking about technical details, at least in this stage. However, in most small to mid-size projects as the business design is ramping up, software architects begin working on the technical architecture of the project. At the end of the business design phase and prior to having the design completed, your architect should have a good understanding of the different technical requirements for the system. An overall architecture for the system can then be created. A technical architecture for the system is very similar to a blue print of a house. It will outline how the system will be designed from a technical perspective, what technology will be used, what data base will be used, as well as performance and security requirements. The goal is to come up with an acceptable architecture prior to the start of the implementation phase.

The first step entails the system architect to design the overall architecture of the system.

VI. Implementation

Most likely you will hand the implementation phase to another team, either your technical staff or an external firm you are outsourcing with. The purpose of this section is to give you an overall picture of the processes followed and discuss some issues that might have an impact on your final application.

Some managers do not want to bother with the details of the implementation phase. A common question is ‘why do you need to know the details of the implementation phase?’ The answer is simple. I believe you will be more capable of making better decisions for your project if you have a good understanding of its technical side. Of course, no one expects a business manager to know all the details of the technology. However, one should be concerned with the overall picture of the implementation phase.

How is the implementation phase divided?

Just like the business design stage, the implementation phase goes through several stages. At the start of the implementation phase, technical staff receives the completed business design documents from the business design team.

1. General architecture design

As we mentioned in an earlier section, your project architect should have completed the overall design of the system prior to the start of the implementation phase. Some software companies like to delay creating the overall architecture until the business design is completed. On most projects, however the luxury of waiting is not an option. If the architecture is already completed, it will be worthwhile to spend some time evaluating it again. At this stage, most of the business requirements are completed and some architecture decisions may no longer be valid.

2. Technical screen design

The second stage of the implementation phase is to design each of the screens in the system from a technical perspective. Similar to how you created business design documents for the system in the earlier stage; the technical staff will need to produce technical design documents for each of the screens. These technical design documents will cover the implementation approach to the screen or module. The staff must also come up with a database design that supports the technical and business requirements of this stage. The outcome of this phase is to produce technical design documents which include having sequence diagrams and database ER diagrams.

3. Coding stage

Technical staff will utilize the business and technical design documents for each screen in the system to implement them. In other words, the technical staff will write the code for the screen to do the tasks outlined in your design document.

I must admit this is an overly simplified explanation of the implementation phase but it is nearly impossible to sum up such a long process in a brief article without simplifying things.

4. Unit testing

Unit testing scripts will be the only test scripts developed by the implementation team. Many software companies ask developers to develop unit testing script prior to the writing of the actual code of the screen. By doing so, the unit test scripts are designed to test the system from the perspective of the functionality. For example, if you are writing a unit testing script to test a login screen, you will probably want to test all possible scenarios around that screen. Scenarios include:

1. user inputs correct user name/password
2. user inputs user name/ incorrect password
3. user inputs incorrect user name/ incorrect password
4. user forgets user name
5. user forgets password
6. user tries to submit with an empty user name
7. user tries to submit with an empty password

A unit test script will test for all of these possibilities. Whether the actual code handles all of these possibilities correctly will be uncovered when the unit test script is run.
Unit testing scripts can either be written in plain English or they can be written as programs that automate the testing process of the system. Unit testing is normally done for each function as the code for it is completed.

Are all the modules implemented at the same time?

That is a tough question to answer but most of the time, the answer is no. If you have 60 modules in the system, it is unlikely to start or complete the business design of all of these modules at the same time. In my experience, whenever my clients have heard this, their response is usually the same. “We were limited on the number of resources we had during the design phase. Why can’t put more people on the project so we can finish faster.” That is a misconception! It reminds me of a saying I once heard, that nine women cannot have a baby in one month.

Deciding the order of which screens are implemented first can have a great impact on the success of your project. Assume that screen “A” depends on information collected from screen “B”. In an ideal world, it makes sense not to develop screen A until screen B is ready. Keep in mind that integration points between the different modules can be one of the main reasons for the success or failure of a project. Thus, it is very important to work with your software architect to decide on the order of implementation.

Before you implement, what is the right technology?

Let’s agree on few things. The earth is round, the sun rises from the east and software engineers can never agree on which technology to use for a project. Software engineers are generally trained in one or two technologies. By its nature, technology requires specialization. So, most likely, your technical staff used the same technology on many different projects. They will try to convince you that that technology is the greatest. The truth is that technology is merely a tool. The final goal is a successful project. If the right tool is not used, your project might suffer. Before we discuss the different technologies, there are two points you will hear technical staff discussing.

1. Open source vs. commercial software, overview

On the one hand, most of us buy windows XP and use it on a daily basis. It is commercial software that you have to pay for a license to use it. Open source technologies are on the other end of the spectrum. Open source technologies are available for the public to use. Sometimes there is a minimal charge for using an open source product in a commercial project.

2. Open source vs. commercial software, quality

There is a perception that open source projects are not of the same quality as commercial products. I am convinced that the quality of the product is not greatly affected by whether the software is an open source or a commercial product. Despite common misconception, there are many open source products that are successfully used all over the world on countess number of projects. Apache.org is a great example of a wide variety of open source products.

3. Open source vs. commercial software, support

Support is the biggest issue that stops companies from adopting certain technologies. Many programmers who write open source software are not making money on it. Thus, they are not necessarily inclined to provide support for it. The more an open source package is used by programmers, the more support available. There are also firms that provide support for open source projects. For example, mysql, an open source database, provides businesses the opportunity to purchase support.

The different types of programming languages

Code is written in a programming language. The language can take different formats. A PC or a MAC understands what is referred to as machine language. You can think of a machine language as a collection of 0s and 1s. Programmers do not write the code in machine language. Instead they write the code in one of two types of programming languages:

Compiled languages: In order for a computer to understand the code, that code must be translated into what is known as a machine language. Compiled languages use a compiler to translate them into machine code. Compiled languages are generally less flexible but still popular.

Interpreted languages: These are computer programming languages created to shorten the traditional programming process. Instead of translating the code into machine language an interpreter is used to execute the code and run it. Interpreters are generally slower to run, but more flexible than compilers.

Here are some of the main languages used to implement web based projects:

1. PHP

PHP is an open source, language. Originally designed as a high level scripting language for producing dynamic Web pages, PHP is used mainly in web-based projects.

Pros:

• PHP provides your project with fast development time and great flexibility.
• Widely accepted - Finding PHP developers is an easy task.
• Cost for development is lower compared to other languages.
• Minimal software cost 

Cons:

• The biggest shortcoming of PHP is its great flexibility! PHP gives the programmer many ways of doing one task. Maintenance of PHP code can be quite a complex task.
• Not built for enterprise level applications. Most companies will not use PHP if they are building a commercial complex application that will be used for years to come. (On the other hand, if you are building a web site that does not require complex logic, PHP might be the right answer for you. Many people will debate this point. Dejanews and Ebay were first build on PHP. They used it for years.) 

2. Java based solutions

Java is a programming language developed by James Gosling and colleagues at Sun Microsystems in the early 1990s. Java borrows much syntax from previous successful languages such as C and C++ . Java based design forces software engineers towards concepts of code reusability.

Pros:

• Built for enterprise level applications. Most companies will use Java based technologies if they are building a commercial complex application that will be used for years to come.
• A well designed, well implemented Java based system is easy to maintain.
• Widely accepted and supported by the open source community.
• Minimal software cost. 

Cons:

• Java based projects can be complex.
• Development time can be longer than other languages.
• Cost of development  is higher than that of PHP.

3. Microsoft .Net

".Net" is a commercial language developed by Microsoft. Dot net, as its pronounced, is a natural evolution from earlier languages developed by Microsoft including VB. “.Net” is also considered Microsoft’s response to java based technologies.

Pros:

• Built for enterprise level applications. Most companies will use .Net based technologies if they are building a commercial complex application that will be used for years to come.
• A well designed, well implemented .NET based system is easy to maintain.
• Minimal software cost 

Cons:

• Not supported by the open source community,
• Net based projects can be complex.
  
• Cost of development is higher than that of PHP.
• Software cost is higher than other languages.

4. Ruby

Ruby is another open source language. Although Ruby has been around since 1995, it has only gained popularity in the last two years. Ruby is a dynamic language that is a blend between a few languages such as Perl, Smalltalk, Eiffel, Ada, and Lisp. Here is a nice blog that explains why you should choose Ruby for your project.

Pros:

• Ruby provides your project with fast development time and great flexibility.
• Ruby is gaining language and support on daily basis.
• Cost for development is lower compared to other languages.
• Minimal software cost.

Cons:

• The language is not widely adopted by software companies. Although sun micro system is making a huge investment by supporting Ruby.

The right technology: In the final analysis

For each language or technology that we mentioned above, there are positives and negatives. Each one of these languages does well in a certain environment. To choose the right technology, you must start with your requirements and by evaluating your application based on the following factors:

• Complexity of application
• Number of users
• Time to develop
• Desktop/web based
• Application to be used for years
• Add new features to the application in the future
• Minimal software cost
• Development cost
  

Using the chart below, you will have a total of 100 points to divide among the different factors. The maximum points a factor will receive is 20 points implying an extremely heavy factor; the lowest number of points is 0 implying a factor has little impact on your project.

Now, you use the points you came up with above to choose the right language. If any of the following statements describes your application, take the points you assigned to the question above and write the value in the box for the language marked with X.

Therefore,  if I have an application that I can describe as Complex, having a small number of users, with time to develop being extremely important, and the application to be used for years to come, I need to have minimal software cost, and I must have lower development cost.

The higher a language scores based on this model, the more suited the language is for your needs. According to this model, .NET is probably not a good option of you. Your best option will be to consider PHP or Ruby.

This should be a starting point. There are many other factors that the model does not take into account. For example, if you plan on hiring staff to complete the project, you need to factor in what kind of experience they have.

Outsourcing my project

This is an important question that you must answer. It is rare to work on a large scale technical project nowadays without working with engineers from India. Whether you need to complete a simple website for your business or you want to write an application that you can sell, India is too close to ignore. Technical staff in India is much cheaper to hire than those in the US. However, you should never make your decision based solely on cost considerations. For a small company, sites like rentacoder.com, allow you to hire a graphic designer or an engineer for minimal cost. Larger more established Indian companies are always ready to offer support to any company willing to pay for their service.

Challenges in working with Indian companies:

• Language barriers: Despite the face that most engineers in India speak English, language can still be a problem on outsourced projects. I kept an email I received from an Indian engineer describing a bug he had with a screen in the system. “When the user press a button from above, he will disappear from within.” I still laugh when I read the bug description. I am sure that the engineer in India did not mean to say that our system was so powerful that it made the user disappear. Engineers in India will be relying mostly on your design documentation to develop the system. Your design documents are written in English. Language by its nature can take on different meanings. Companies in the US that work face to face disagree on the meaning of a design document, so imagine the confusion that can result when working with a company in India.

• Design Documentation: This is a simple one, if you are going to outsource, your design documents, including requirements must be nearly complete. That will save you and the company you are hiring much trouble.

• Time: I always argue that if you need to get a project completed faster, you should not outsource it. Outsourcing by its nature will bring on many issues that you will have to resolve. Outsourced projects always end up taking longer because they tend to have a higher number of issues surrounding them.

• Cost: If cost is a big factor, then outsourcing is an alternative that is worthwhile to consider. I believe that companies can save between 25 to 40% in development cost if they choose to outsource.

• Length of Use: If you have technical staff on hand, you should get them involved in the project. If you are going to use the application and market it, you want to make sure that someone from your companies is actively involved in the technical aspect of the process. Outsourcing makes that difficult to do.

• Complexity:  The less complicated an application is, the easier it is to outsource. Making a small change to a flash header, for example is something I would outsource without a problem. The more complex the application is, the more I am hesitant to outsource.
  I am not against outsourcing; however I want you to be aware of some of the issues related to outsourcing. The truth is that companies outsource projects successfully on a daily basis.
  

Two main factors help: Either the projects are small or limited in scope, or they can be very complex but the company has the staff to manage the project and its risks.

VII. QA phase

QA Phase is where you start testing the application to make sure it meets the different requirements. This slide gives a general introduction to some of the main concepts around testing.

Levels of testing:

As the application is getting ready to be released, there are many types of testing that must take place. Each of these types outlined below tests a different portion of the application during the testing stage.

Functional verification

This type of testing will be completed by either a test team or by the business design team. Functional test scripts should be developed prior to completion of the implementation phase. They also must be developed by your own staff. Whatever you do, do not rely on technical staff to write the test scripts! When an engineer writes test scripts, he bases his script on how his code expects the application to act. Good scripts should test the application based on how the business design documents expect the application to act. This slide gives an excellent view of testing software.

There are different levels of test scripts:

• Screen test scripts should be developed for each of the screens in the system. These scripts test the user interaction with a screen.
• Module and integration test scripts test the integration between the different parts of the system.
  

As functional testing is completed for each module in the system, you will discover bugs in the system. As bugs are discovered, the implementation team will fix these bugs and you will have to run the functional testing script for that module again. A module is successfully tested after a test script is run again and no bugs are found.

Performance testing

The purpose of this type of test is to validate how the system performs in general and how it will perform when dealing with a large number of users. The importance of performance testing varies from one application to another. If you develop an application that will be used by 5 users, you probably will not need to test how that application will perform if 1000 users are using it at the same time. The larger and more complex the application is, the more critical performance testing becomes. You should have already developed and agreed on general performance guidelines with the implementation team.

Usability testing

This type of testing tests the application from the perspective of the user. Is the application user friendly? How is the user going to interact with the application? Let’s assume that your software has a checkout process. Can you imagine a check out process that involves 10 steps?! One step asks the user to input his first and last name and then press “submit” on the screen. Step two, asks for his address and telephone and then he has to press “submit” on the screen again. Step three will ask for his email and then he has to press “submit” on the screen and so on. Most users would find this an annoying and lengthy process that is unnecessary and should be fixed.

Requirements traceability and testing

This is an extremely important exercise. You need to take each of the requirements in your requirement document and test to see if the system performs that requirement. Each missing requirement constitutes a gap in the application that must be accounted for.

User acceptance testing

This is the last type of testing that takes place in the QA phase. The purpose of this testing is to give the final go ahead for the application. In general, the project should not move into User acceptance testing (UAT) unless all functional testing is successfully completed. UAT will most likely use some of the functional testing scripts used above to go back and test some of the modules in the system. The purpose of this testing is to provide a final sanity check to ensure that the software does indeed match all the requirements.

VIII. General thoughts around software projects:

Pitfalls of software projects:Process for the sake of the process

Some people are fascinated by a software process that they forget the real purpose of it. Process is there to help you complete your project on time and within budget. It is a means to an end. Process is not a goal in itself. That might sound silly to some people but the truth is that many software managers are so stuck on the actual process that they forget the real goal of actually delivering software. I have worked on projects where the actual process became a barrier in the way of doing the work. In one instance, the process was so developed and outlined that you could not write a single line of code without having to write 5 or so pages with every minute detail. On another project, management insisted on perfecting every detail on each phase before we could even begin on the next. Although perfection is nice, it is not necessary to spend countless hours spell checking design documents or ensuring that the margins on documentation was perfect. Instead of completing the design phase in one month, it took six months! Perhaps these are extreme cases, but any software engineer will surely have their share of horror stories to tell.

Pitfalls of software projects: the anti process

The other extreme is the anti-process mentality. This is a pitfall many software engineers encounter. As software engineers, we love to code. We enjoy spending time in the implementation phase. That is our passion and that is where we shine. But that means that sometimes we do not appreciate the importance of the other phases of software projects. I found that engineers can be very convincing when arguing against “wasting” time on requirements gathering or testing an application. Managers are trying to cut down on cost and finish the software project faster so they are receptive to anything that can help them do so. The truth is that cutting time on requirements gathering or quality assurance phases can be disastrous! Software projects are full of stories of an application not fully tested and after users start using it, someone has to pull the plug on the application. A site is shutdown, an application is uninstalled and more time (and more money) must be spent to fix the application.

Web Application vs. Web site

The difference between a "web site" and a "web application" is subjective. Although some of the technologies are the same, web applications tend to be far more interactive and more difficult to create than typical web sites. For example, a web site is mostly read-only, with occasional forms for submitting information. For this, simple technologies such as HTML combined with JavaServer Pages (JSPs) can do the job. A web application, on the other hand, is typically a custom application intended to perform a specific business or technical function. They are often written as replacements for existing systems in an effort to enable browser-based access. When replacing existing systems, developers are typically asked to duplicate all of the existing functionality, using a web browser and HTML. This is difficult at best because of HTML's limited support for sophisticated GUI components. Most of the screens in a web application are dynamically generated and customized on a per-user basis, while many pages on a typical web site are static.