Wednesday, March 18, 2009

Python Notes – 2 : Variables, Statements, Expressions, Operators, and Functions

Welcome to our second note in our Python learning process. In this note we will talk about variables, statements, expressions, operators, comments, and functions. These are the very basic building blocks of you programs whatever its final size

Variables

The assignment statement creates new variables and gives them values

>>> message = "What's up, Doc?"

>>> n = 17

>>> pi = 3.14159

Python is a dynamically typed language. Which means that variables' types don't have to be defined before the variables use. The python interpreter figure out what type a variable is when you first assign it a value.

Variables naming

  • Variable name can contain letters, numbers, and underscore.
  • Variable name must begin with letter.
  • Variable names are case-sensitive ( x different than X ).
  • Variable names can't be one of the Python language reserved words, which are :
and continue else for import not raise
assert def except from in or return
break def exec global is pass try
class elif finally if lambda print while

Statements

A statement is an instruction that the Python interpreter can execute. A Python script is a sequence of statements. The results appear one at a time as the statements execute.

For example, the script

>>> print 1

>>> 1

>>> x = 2

>>> print x

>>> 2

Expressions

An expression is a combination of values, variables, and operators. If you type an expression on the command line, the interpreter evaluates it and displays the result:

>>> 1 + 1

2

Boolean Expressions

A boolean expression is an expression that is either true or false. Boolean expressions are expressions that uses logical operators. There are three logical operators: and, or, not. The operands of logical operands should be boolean expressions. In Python there is no boolean data type instead 0, '', [], (), {}, and None are false in a boolean context; everything else is true. The absence of boolean data types results in the following behavior of logical operators:

  • All the behaviors depends on the order of expressions evaluation, which is from left to right.
  • and : returns the first false value, if all values are true, returns the last value.

>>> 'a' and 'b'

'b'

>>> ' ' and 'b'

' '

>>> 'a' and 'b' and 'c'

'c'

  • or : returns the first true value, if all values are false, returns the last value

>>> 'a' or 'b'

'a'

>>> ' ' or 'b'

'b'

>>> ' ' or [] or {}

{}

  • not : evaluates the boolean expression that follows it and inverts it whole value.

>>> not (1 and 0)

True

>>> not (1 or 0)

False

Operators and operands

Operators are special symbols that represent computations like addition and multiplication. The values the operator uses are called operands.

The symbols +, -, and /, and the use of parenthesis for grouping, mean in

Python what they mean in mathematics. The asterisk (*) is the symbol for multiplication, and ** is the symbol for exponentiation.

Order of operations

When more than one operator appears in an expression, the order of evaluation depends on the rules of precedence. The following is operators from highest precedence to lower precedence:

  • Parentheses
  • Exponentiation
  • Multiplication / Division
  • Addition / Subtraction

Operators with the same precedence are evaluated from left to right.

Comments

Comments start with #.

Built-in Functions

Here are some of the built-in functions available in Python:

  • Type conversion functions which converts values from one type to another. Like:
    • int() :

>>> int("32")

32

>>> int("Hello")

ValueError: invalid literal for int(): Hello

    • float():

>>> float(32)

32.0

>>> float("3.14159")

3.14159

    • str():

>>> str(32)

'32'

>>> str(3.14149)

'3.14149'

  • Math functions which provides most of the familiar mathematical functions. These functions are implemented in a module called math, we will talk about modules soon. But for now, if you want to use these functions, you have to import the math module. This is very straight foreword, using following statement
    • >>> import math
    • After the import statement, you can simply call the math functions.

>>> decibel = math.log10 (17.0)

>>> angle = 1.5

>>> height = math.sin(angle)

>>> degrees = 45

>>> angle = degrees * 2 * math.pi / 360.0

>>> math.sin(angle)

0.707106781187

>>> math.sqrt(2) / 2.0

0.707106781187

There are many other built-in functions, we mentioned these two types just as examples.

Creating your own functions

A function is a named sequence of statements that performs a desired operation. This operation is specified in a function definition.

The syntax for a function definition is:

def FUNCTION_NAME( LIST OF PARAMETERS ):

STATEMENTS

  • def is a keyword.
  • FUNCTION_NAME could be anything except Python keywords.
  • LIST OF PARAMETERS are the ordered list parameters the function needs to operate. The empty parentheses indicate no parameters needed.
  • The function body doesn't have starting or ending characters (like "{" and "}" in C like languages ). Indentation is the only scope identifier for statements in Python. The first statement that doesn't follow the indentation, will be considered out of the function and will mark the function termination.

# just to show the function scope and structure

def sample_func():

    Print "we are in the function body scope"

    Print "we still in the function scope"

Print "we are out of the function scope"

# another function with parameters

def sum(x, y):

    print x + y

  • To call a function just use the function name followed by the list of parameters between parentheses

>>> sample_func()

>>> sum(1,2)

3

  • Functions can call other functions in their body.
  • Functions can return values to its caller

def sum(x,y):

return x +y

def print_sum(x,y):

print sum(x,y)

print_sum(2,3)

5

  • When you create a local variable inside a function, it only exists inside the function, and you cannot use it outside. The following will generate a run-time error when trying to access Z variable.

def sum(x,y):

Z = x +y

print Z # will generate a runtime error

Returning multiple values

def divide(a,b):

q = a/b

r = a - q*b

return q,r

>>> x,y = divide(42,5) # x = 8, y = 2

>>> x

8

>>> y

2

In this note we talked about variables, statements, expressions, operators, comments, and functions. These are the very basic building blocks of you programs whatever its final size. We will continue our learning in the upcoming notes.

Python Notes – 1 : Setup

I write this notes about Python during my learning process of it. I'm not an expert in Python. I'm just writing this notes while I learn Python aiming to help others who are learning it too. Never hesitate to comment on anything you want.

Origins of Python

  • Developed by Guido van Rossum.
  • Derived from ABC, a teaching language that was developed in 1980s.

Why Pyhton ?

  • Python code is pretty simple, compact and easy to learn.
  • Code simplicity let you focus on the core program functionality.
  • Suitable for programming languages introductory courses.
  • Provides a balance between the practical and the conceptual. You start writing after the first tutorial; and if you want to go further in advanced topics you will find a large library of modules that can be used to do all sorts of tasks ranging from web-programming to graphics.
  • Python borrows features from both functional programming languages and object-oriented programming languages, which enables it to serve as an excellent foundation for introducing important computer science concepts.
  • Python allow you to see higher level of success and a lower level of frustration :)

How to setup Python ?

Python is free downloadable from many source, and it have also many IDEs and editors. Aiming to start coding quickly without wasting time in IDE exploration, setup, configuration problems; we will use just the ActivePython. Let's see how to do that:

  1. Download ActivePython, it is free.
  2. After you install ActivePython you will got that the following change in your start menu. We will not use Python Intercative Shell nor PythonWin Editor in our first steps, we will use them in upcoming notes.

      clip_image001

  1. In our first note we will use Python from the Windows Command window. To do that we need to add the Python installation path to the Environment Variables of the operating system.
  2. If you are using Windows XP or Windows 2000 do the following (if you are using Windows Vista go for step 5):
    1. Right-click My Computer and select Properties from the displayed context menu. You will got the System Properties dialog

      clip_image002

  1. Then click the Advanced Tab

      clip_image003

  1. Then click the Environment Variables button near the left bottom. You will got the Environment variables dialog

      clip_image004

  1. Select the variable path from the second list called System Variables. Then click Edit button, you will got the Edit System Variable dialog box

      clip_image005

  1. Append the Python installation path to the string in Variable value textbox. The default installation path for ActivePython in our case is C:\Python25\; the semicolon ';' is not part of the path but is mandatory to be added at the end. This semicolon acts as the delimiter that enables the operating system to differentiate between this list of paths.
  1. If you using Windows Vista, do the following (very similar steps):
    1. Right-click My Computer and select Properties from the displayed context menu. You will got the System Properties dialog

      clip_image006

  1. Then the link Advanced system settings on the left menu, you will got the System Properties dialog box

      clip_image007

  1. Then click the Advanced tab

      clip_image008

  1. Click the Environment Variables button near the right bottom, you will got Environment Variables dialog box

      clip_image009

  1. Select the variable path from the second list called System Variables. Then click Edit button, you will got the Edit System Variable dialog box

      clip_image005[1]

  1. Append the Python installation path to the string in Variable value textbox. The default installation path for ActivePython in our case is C:\Python25\; the semicolon ';' is not part of the path but is mandatory to be added at the end. This semicolon acts as the delimiter that enables the operating system to differentiate between this list of paths.

Now, we have Python installed and configured on our machines.

Let's begin in Python

Python is a high-level programming language. It is interpreted language, which means that its programs are executed by an interpreter rather than a compiler. There are two ways to use the interpreter:

  • Command-line mode where you type your python programs at the command-line and the interpreter prints the result.
    • Open the windows command window and type python, to invoke the python interpreter.
    • Type print "Hello world !" and press enter. Guess what, you have wrote your first program in Python :)

       clip_image010

  • Script mode where you write your program in a file and use the interpreter to execute the contents of the file. Such a file called script. By convention python files have extension .py.
    • To try that mode, create a file script1.py in your Python installation folder ( default is : C:\Python25 ).
    • In this file write print "Hello world !" , save and close.
    • Open the windows command window and type python, to invoke the python interpreter.
    • Type import script1 and press enter. You just ran your first Python program.

       clip_image011

We will use the script mode in most of our notes.

This is just the beginning of our notes about Python. Now we have Python installed and configured on our machines. We will continue our learning in the upcoming notes.

Wednesday, March 4, 2009

Type checking in programming languages

To discuss type checking in programming languages we have to define type checking. Simply,Type checking is how type errors are checked.

Type checking could happen at compile-time or at run-time. When type checking happens at the compile-time it is called Static Type Checking. When type checking happens at the run-time it is called Dynamic Type Checking.

Programming Languages differs in

  • when they check types.
  • Were it enforces types or not.

Based on these two factors, programming languages could be classified from the type checking perspective into the following classes:

Statically Typed Languages

In these languages, types are fixed at compile time. Most statically typed languages enforce this by requiring you to declare all variables with their data types before using them

C, Java, and C# are statically typed languages.

Dynamically Typed Languages

In these languages, types are discovered at execution time; the opposite of statically typed languages. These languages figure out what type a variable is when you first assign it a value.

VBScript and Python are dynamically typed languages.

Strongly Typed Languages

In these languages, types are always enforced. If you have an integer, you can’t treat it like a string without explicitly converting it.

Java and Python are strongly typed languages.

Weakly Typed Languages

In these languages, types may be ignored; the opposite of strongly typed languages. In VBScript, you can concatenate the string ‘12’ and the integer 3 to get the string ‘123’, then treat that as the integer 123, all without any explicit conversion.

VBScript is weakly typed language.

Programming languages could be classified based on many other factors and differences which we may discuss in further posts.

Thursday, February 26, 2009

Introduction to Mining Software Engineering Data

 

Tao Xie | North Carolina State University | xie@csc.ncsu.edu

Ahmed E. Hassan | University of Victoria | ahmed@uvic.ca

Part I

Mining Software Engineering Data goals:

  • Transform static record-keeping SE data to active data.
  • Make SE data actionable by uncovering hidden patterns and trends.

Uses of mining SE data:

  • Gain empirically-based understanding of software development.
  • Predict, plan, and understand various aspects of a project.
  • Support future development and project management activities.

clip_image001

Types of SE Data:

  • Historical data

Used primarily for record-keeping activities (checking the status of a bug, retrieving old code)

  • Version or source control:
    • cvs, subversion, perforce.
    • Store changes to the data
  • Bug systems:
    • bugzilla, GNATS, JIRA.
    • Follow the resolution of defects.
  • Mailing lists:
    • Mbox
    • Record rationale for decisions throughout the life of a project.
  • Multi-run and Multi-site data
    • Execution traces
    • Deployment logs

clip_image002

Software Maintenance Activities

  • Perfective: add new functionality
  • Corrective: fix faults
  • Adaptive: new file formats, refactoring

    Source Control Repositories

    A source control system tracks changes to ChangeUnits.

    Examples of ChangeUnits:

    • File
    • Function
    • Dependency (e.g. Call )

    For each ChangeUnit, it tracks the developer, time, change message, co-changing Units.

    clip_image003

    Change Propagation

    clip_image004

    Measuring Change Propagation

    clip_image005

    clip_image006

    We want:

    • High precision to avoid wasting time
    • High recall to avoid bugs

    Guiding Change Propagation

    Mine association rules from change history.

    Use rules to help propagate changes:

    • Recall as high as 44%
    • Precision around 30%

    High precision and recall reached in < 1mth

    Prediction accuracy improves prior to a release (i.e., during maintenance phase)

    Code Sticky Notes

    Traditional dependency graphs and program understanding models usually do not use historical information.

    Static dependencies capture only a static view of a system - not enough detail!

    Development history can help understand the current structure (architecture) of a software system.

    Studying Conway's Law

    "The structure of a software system is a direct reflection of the structure of the development team"

    clip_image007

    Predicting Bugs

    Studies have shown that most complexity metrics correlate well with LOC ! (Lines of Code)

    Noteworthy findings:

    • Previous bugs are good predictor of future bugs.
    • The more a file changes, the more likely it will have bugs in it.
    • Recent changes affect more the bug potential of a file over older changes (weighted time damp models)
    • Number of developers is of little help in predicting bugs.
    • Hard to generalize bug predictors across projects unless in similar domains.

    Example 1 : using imports in Eclipse to predict bugs

    • 71% of files that import compiler packages, had to be fixed later on.
    • 14% of all files that import ui packages, had to be fixed later on.

    Example 2 : don't program on fridays

    Percentage of bug-introducing changes for eclipse, most high in Friday.

    Classifying changes as Buggy or Clean

    Given a change can we warn a developer that there is a bug in it ?

    • Recall/Precision in 50-60% range.

    Project Communication - Mailing lists

    Most open source projects communicate through mailing lists or IRC channels.

    Rich source of information about the inner workings of large projects.

    Discussion cover topics such as future plans, design decision, project policies, code or path reviews.

    Social network analysis could be performed on discussion threads.

    Social Network Analysis

    Mail list activity

    • Strongly correlates with code change activity.
    • Moderately correlates with document change activity.

    Social network measures (in-degree, out-degree, between's) indicate that committers play much more significant roles in the mailing list community that non-committers.

    Immigration rate of developers

    When will a developer be invited to join a project?

    • Expertise vs. interest

    The patch review process

    Two review styles

    • RTC : Review-then-Commit
    • CTR : Commit-then-Review

    80% patches reviewed within 3.5 days and 50% reviewed in < 19 hrs

    Measure a team's morale around release time

    Study the content of messages before and after release.

    Use dimensions from a psychometric text analysis tool.

    Program Source Code

    Code Entities

    clip_image008

    Mining API Usage Patterns

    How should an API be used correctly?

    • An API may serve multiple functionalities --> Different styles of API usage

    "I know what type of object I need, but I don't know how to write the code to get the object"

    • Can we synthesize jungloid code fragments automatically?
    • Given a simple query describing the desired code in terms of input and output types, return a code segement.

    "I know what method call I need, but I don't know how to write code before and after this method call"

    Relationships between Code Entities

    • Mine framework reuse patterns
      • Membership relationships
        • A class contains membership functions
      • Reuse relationships
        • Class inheritance / instantiation
        • Function invocations / overriding
    • Mine software plagiarism
      • Program dependence graphs

    Program Execution Traces

    Method-Entry/Exit States

    Goal: Mine specifications (pre/post conditions) or object behavior (object transition diagrams)

    State of an object: values of transitively reachable fields.

    Method-entry state: Receiver-object state, method argument values.

    Method-exit state: Receiver-object state, updated method argument values, method return value.

    Other Profiled program states

    Goal: detect or locate bugs.

    Values of variables at certain code locations

    Object/static field read/write

    Method-call arguments

    Method returns

    Sampled predictions on values of variables

    Executed Structural Entities

    Goal: Locate bugs.

    Executed branches/paths, def-use pairs.

    Executed function/method calls.

    Group methods invoked on the same object

    Profiling options

    Execution hit vs. count

    Execution order (sequences)

    Part II

    How can you mine Software Engineering data?

    Overview of data mining techniques

    Association rules and frequent patterns

    Classification

    Clustering

    Misc.

    Association Rules

    Example:

    Finding highly correlated method call pairs.

    Check the revisions (fixes to bugs), find the pairs of method calls whose confidences have improved dramatically by frequent added fixes.

    Those are the matching method call pairs that may often be violated by programmers

    Conflicting Patterns

    999 out of 1000 times spin_lock is followed by spin_unlock

    The single time that spin_unlock does not follow may likely be an error.

    We can detect an error without knowing the correctness rules.

    Detect Copy-Paste Code

    Apply closed sequential pattern mining techniques.

    Customizing the techniques:

    • A copy-paste segment typically does not have big gaps.
      • Use a maximum gap threshold to control.
    • Output the instances of patterns ( i.e., the copy-pasted code segments) instead of the patterns.
    • Use small copy-pasted segments to form larger ones.
    • Prune false positives: tiny segments, unmappable segments, overlapping segments, and segments with large gaps.

    Find Bugs in Copy-Pasted Segments

    For two copy-pasted segments, are the modifications consistent?

    • Identifier a in segment S1 is changed to b in segment S3 3 times, but remains unchanged once - likely a bug
    • The heuristics may not be correct all the time

    The lower the unchanged rate of an identifier, the more likely there is a bug.

    Mining Rules in Traces

    Mining association rules or sequential patterns S --> F, where S is a statement and F is the status of program failure.

    The higher the confidence, the more likely S is faulty or related to a fault.

    Using only one statement at the left side of the rule can be misleading, since a fault may led by a combination of statements.

    Frequent patterns can be used to improve.

    Mining Emerging Patterns in Traces

    A method executed only in failing runs is likely to point to the defect.

    Comparing the coverage of passing and failing program runs helps.

    Mining patterns frequent in failing program runs but infrequent in passing program runs.

    Sequential patterns may be used.

    Classification

    Classification: A 2-step Process

    Model construction: describe a set of predetermined classes

    • Training dataset: tuples for model construction
      • Each tuples/sample belongs to a predefined class
    • Classification rules, decision trees, or math formulae

    Model application: classify unseen objects

    • Estimate accuracy of the model using an independent test set.
    • Acceptable accuracy --> apply the model to classify tuples with known class labels.

    Supervised learning (Classification)

    • Supervision: objects in the training data set have labels
    • New data is classified based on the training set

    Unsupervised learning (Clustering)

    • The class labels of training data are unknown
    • Given a set of measurements, observations, etc. with the aim of establishing the existence of classes or clusters in the data.

    GUI-Application Stabilizer

    Given a program state S and an event e, predict whether e likely results in a bug

    • Positive samples: past bugs
    • Negative samples: not bug reports

    A K-NN based approach

    • Consider the k closest cases reported before
    • Compare sum 1/d for bug cases and not-bug cases, where d is the similarity between the current state and the reported states.
    • If the current state is more similar to bugs, predict a bug.

    Clustering

    What is clustering ==> group data into clusters.

    Similar to one another within the same cluster.

    Dissimilar to the objects in other clusters.

    Unsupervised learning: no predefined classes.

    Clustering and Categorization

    Software categorization

    Partitioning software systems into categories

    Categories predefined - a classification problem

    Categories discovered automatically - a clustering problem

    Software Categorization - MUDABlue

    Understanding source code

    • Use Latent Semantic Analysis (LSA) to find similarity between software systems.
    • Use identifiers (e.g., variable names, function names) as features
      • "gtk_window" represents some window
      • The source code near "gtk_window" contains some GUI operation on the window.
    • Extracting categories using frequent identifiers
      • "gtk_window", "gtk_main", and "gpointer" --> GTK related software system
      • Use LSA to find relationships between identifiers

    Other Mining Techniques

    Automation/grammar/regular expression learning

    Searching/matching

    Concept analysis

    Template-based analysis

    Abstraction-based analysis

Sunday, February 22, 2009

Center of Innovation & Competitiveness (INCOM) - Nile University

Nile University Center for Innovation and Competitiveness (NU/INCOM) is primarily focused on identifying, researching and promoting innovation practices that have improved competitiveness at the company, industry and country levels, with special emphasis on Egypt and the MENA region. Innovation is one of the most important competitive priorities in firms and in nations, and it is a major driving force for change in today's world. It is critical in the formulation of successful manufactoring strategies for nations, at the micro and macro levels, and in enhancing their economic development and global market positioning. Innovation guides any business, small, medium or large, in its ability to successfully compete in the global market, thus impacting the international competitiveness of firms and nations.

Strategic policy planning and implemention in modern governments requires technological foresight and database and system modeling capabilities upon which policies and initiatives for national priorities can be determined. In this area INCOM will focus on developing database and modeling tools to aide government agancies in:
  • Developing technology foresight capability
  • Policy analysis capability
  • Strategic policy evaluation capabilities

Strategies for competitiveness in business firms require efficient and effective use of technological and business resources and keen understanding of the global market and its dynamics. CIC will focus on the firms' basic competitive priories including:

  • Providing innovative products/services that compete favorably with competition.
  • Predictive benchmarking of competitor's products and services.
  • Producing products/services with high quality performance standards.
  • Producing and distributing products/services at a competitive price.
  • Meeting delivery scheduling and reacting quickly to customer schedule changes.
  • Reacting to changes in market needs and in product requirements.
  • Offering a broad platform of services to boost customer satisfaction.

Through NU Executive Development Center, INCOM will also provide training and research services to encourage and facilitate entrepreneurship capabilities in small, medium and large business enterprises in a global context.

Thursday, February 5, 2009

Bug Counts vs. Test Coverage

What to Do When Bug Counts Don’t Speak for Themselves?
Bug counts on a project speak volumes about the quality of testing for a particular product and how vigorous the test team is working to "assure quality." Bug counts are invariably a primary area of test metrics that are reported to management. What is the rationale behind drawing so much attention to the number of bugs being found through the course of a project?
I have heard it said that QE’s job is to find bugs. If this is the assumption of management, bug counts will be an important indicator to them that QE is doing its job. They expect to see bug counts rise dramatically in the early stages of testing, and they expect to see the find rate decrease as the project comes to an end. These are management’s statistical expectations when they believe bug counts are a metric to assess quality of testing.
If high bug counts, then, are an indicator that quality is going up, low bug counts can be seen as an indicator that something just isn’t right with the testing process. Management might imagine different problems that are preventing bugs from being found:

  • Test coverage isn’t complete; maybe major areas of functionality aren’t being tested.
  • Testing is only scratching the surface of all functionality, not digging in to the real complexities of the code.
  • Our testers just aren’t that good.

Management might see red flags when bug counts are low, but a number of causes may contribute to low bug counts. On the second or third iteration of a product, the bulk of the defects may have been found on an earlier cycle. Or especially good development practices may have been implemented: strong unit testing, code reviews, good documentation, and not working developers to death. These are supposed to result in lower bug counts.
Ultimately, however, QE will justify low bug counts when it can justify its test coverage. If the product under test is being tested with thorough coverage, the bug count should be treated only as a supporting statistic, not the primary one. After all, we all know that a quality product hasn’t been reached when a certain bug count is reached. Quality is achieved when test coverage is maximized and bug finds decrease to a minimum.
There are several things you can do when bug counts are low and management is questioning the quality of testing:

  1. Take stock. Call a meeting with your test team, go through the areas of test, possibly even some test cases themselves, and get a general feel for how much test coverage you really have. Maybe you’ll discover that an area of test really is being missed. Perhaps there is some misunderstanding of who should be testing what and some functionality fell between the cracks. Brainstorm more testing methods and techniques, and generate ideas of how your team can broaden the testing efforts. Before going to other groups or departments, get a solid understanding of where your team is in the process.
  2. Talk to development. Go over your current test coverage with development, and see if they have any input on areas you might also investigate. Ask them what the trouble spots are, if they can suggest lower-level tests that may ferret out more bugs, and possibly even conduct a test case review with them. On my last project, we sent out the test cases of a certain functionality to the appropriate developer for review. Though many times developers can be reluctant to help testers, demonstrate to them that it is in their best interest that we thoroughly test their code—if it’s solid, they have nothing to worry about.
  3. Communicate with management. When bug counts are low, use test coverage to justify them. This doesn’t mean dismissing the fact that the bug count is low. It means using the bug count as an indicator to do some analysis into the testing practices you are doing, and verifying that high test coverage is being achieved. If it is, explain to management your findings. Demonstrate by solid metrics that you are performing thorough testing, that you can’t force bug counts to go up, and that maybe—just maybe—a low bug count means you’ve got a quality product on your hands!

One thing to bear in mind: while you can use the above methods during testing cycles to understand and cope with a low bug count, the ideas are still applicable before testing even begins, while test cases are being written for a project, and while development is still in full swing. Good test coverage is something to be planned ahead of time, and having gone through the effort of mapping coverage and functional test cases early in the project, you will prevent yourself from spending valuable testing cycles repeating tasks.
While low bug counts can cause people in both development and management to question the effectiveness of the testing, do not be defensive about it. Use it as a trigger to prove what you should already know—your testing efforts are appropriate, effective, and your coverage is maximized. Don’t let your bug counts do the talking—your test coverage should say it all.

7 Habits of Highly Insecure Software

Habit # 1: Poorly Constrained Input
By far, the number one cause of security vulnerabilities in software stems from the failure to properly constrain input. The most infamous security vulnerability resulting from this habit is the buffer overflow. Buffer overflows happen when application developers use languages (like C and C++) that allow them to allocate a fixed amount of memory to hold some user-supplied data. This usually doesn’t present a problem when input is properly constrained or when input strings are of the length that developers expected. When data makes it past these checks, though, it can overwrite space in memory reserved for other data, and in some cases force commands in the input string to be executed. Other unconstrained input can cause problems, too, like escape characters, reserved words, commands, and SQL (Structured Query Language) statements.

Habit # 2: Temporary Files
Usually we think of the file system as a place to store persistent data; information that will still be there when the power is shut off. Applications, though, also write out temporary files—files that store data only for a short period and then are deleted. Temporary files can create major security holes when sensitive data is exposed. Common (inappropriate) uses of temp files include user credentials (passwords), unencrypted but sensitive information (CD-keys), among others.

Habit # 3: Securing Only the Most Common Access Route
How many ways could you open a text document in Windows? You could double-click on the file in Windows Explorer; or open your favorite text editor, and type the file name in the open dialog; or type the file name into an Internet Explorer window. The truth is, if you put your mind to it, you could think of at least a dozen ways to open that file. Now imagine implementing some security control on that document. You would have to think of every possible access route to the document, and chances are, you’re likely to miss a few. Developers fall into this dilemma too. When requirements change, or when a new application version is being developed, security controls are often “added-on” to an application. Also, when a security bug is reported, developers may patch the application to fix the particular input sequence reported and still leave other, underused access routes unprotected. The result: the reappearance of supposedly fixed bugs or alternate access routes that bypass security mechanisms.

Habit # 4: Insecure Defaults
We are all guilty of the mortal sin of clicking “Next” or “Finish” on an installation wizard without reading the details and just accept recommended configurations. But is it a sin? The application’s developers and testers know more about the application than we do, so it seems natural not to worry about awkward installation options and just accept defaults. Most users think this way and I can’t say that I blame them. So what does this mean for security-conscious testers? It means that we need to ensure security out of the box. We have to make sure that default values err on the side of security, and that insecure configurations are appropriately explained to users.

Habit # 5: Trust of the Registry and File System Data
When developers read information from the registry, they trust that the values are accurate and haven’t been tampered with maliciously. This is especially true if their code wrote those values to the registry in the first place. One of the most extreme vulnerabilities is when sensitive data, such as passwords, is stored unprotected in the registry. We have found that passwords, configuration options, CD keys, and other sensitive data are often stored unencrypted in the registry—ripe for the reading.

Habit # 6: Unconstrained Application Logic
It’s pretty clear that we need to examine individual functions to make sure that they are secure. If a feature used in a Web browser is not supposed to allow the reading of any file except a cookie, then there’s a pretty good chance that a test case was run to verify that. Features are not likely to be as well constrained when they are combined or when commands are executed in a loop. Constraining loops can be an exquisitely difficult programming task. Many denial of service attacks are made possible by getting some benign function (such as one that writes out a cookie) to execute over and over again and consume system resources.

Habit # 7: Poor Security Checks with Respect to Time
The ideal situation is that every time sensitive operations are performed, checks are made to ensure they will succeed securely. If too much time lapses between time-of-check and time-of-use, then the possibility for the attacker to get in the middle of such a transaction must be considered. It is the old “bait and switch” con applied to computing: Bait the application with legitimate information, and then switch that information with illegitimate data before the application notices.

Using these seven habits as a guideline for your software project will help ensure a successful outcome. There’s no such thing as 100 percent bug free software. Our goal, however, is to get as close as possible.