text:
10.4236/jsea.2017.104020
Estimation Models for Software Functional Test Effort
Kamala Ramasubramani Jayakumar1, Alain Abran2
1Amitysoft Technologies, Chennai, India
2École de technologie supérieure, University of Quebec, Montreal, Canada
Received: January 25, 2017; Accepted: April 24, 2017; Published: April 27, 2017
The International Software Benchmarking and Standards Group (ISBSG) database was used to build estimation models for estimating software functional test effort. The analysis of the data revealed three test productivity patterns representing economies or diseconomies of scale and these patterns served as a basis for investigating the characteristics of the corresponding projects. Three groups of projects related to the three different productivity patterns, characterized by domain, team size, elapsed time and rigor of verification and validation carried out during development, were found to be statistically significant. Within each project group, the variations in test effort can be explained, in addition to functional size, by 1) the processes executed during development, and 2) the processes adopted for testing. Portfolios of estimation models were built using combinations of the three independent variables. Performance of the estimation models built using the function point method innovated by the Common Software Measurement International Consortium (COSMIC) known as COSMIC Function Points, and the one advocated by the International Function Point Users Group (IFPUG) known as IFPUG Function Points, were compared to evaluate the impact of these respective sizing methods on test effort estimation.
COSMIC Function Points, Estimation, Functional Sizing, Performance Measurement, Software Testing
This paper reports on a set of estimation models designed with data chosen from the ISBSG repository consisting of functional sizes reported both in IFPUG function points [1] and COSMIC function points. These estimation models were evaluated using criteria for measuring outputs from estimation models. The models were compared to understand their performance based on the measure of their predictability.
The motivation for this research work arises from the fact that existing techniques for estimating test effort (such as judgment-based, work breakdown, factors & weights, and functional size based techniques) suffer from several limitations [2] [3] , while other innovative approaches for estimating testing effort (such as fuzzy inference, artificial neural networks, and case-based reasoning as proposed in the literature) are yet to be adopted in the industry. There is a growing body of work on the use of the COSMIC function points [4] [5] for estimation and performance measurement of software development projects which can be adapted for estimating software test effort too.
The remainder of this paper is structured as follows. Section 2 presents the data preparation; Section 3 is data analysis; Section 4 is the estimation models and Section 5 is the conclusions.
2. Data Preparation
2.1. ISBSG Data
Release 12 of ISBSG data published in 2013 [6] consists of data related to parameters of software projects re-ported over the last two and half decades, providing industry and researchers with standardized data for benchmarking and estimation. The ISBSG dataset has been extensively reviewed for its applicability to building effort estimation models, including effects of outliers and missing values [7] [8] .
The attributes of interest for test effort estimation models are:
a. Functional size data based on international measurement standards such as IFPUG and COSMIC function points.
b. Schedule, team size, work effort information, project elapsed time and breakdown of work effort by project phase (planning, specifications, design, build, test and install).
c. Project process related data based on software life cycle activities (e.g. planning, specifications, design, build, test) and adoption of practices from standards or models such as ISO 9001, CMMI, SPICE, PSP etc. used in developing the software.
d. Grouping attributes: industry sector, application group (e.g., business, real time etc.), and development type (new development, enhancement or re-deve- lopment).
e. Development platform: PC, mid-range, main frame or multi-platform.
f. Architecture: whether the application is standalone, multi-tier, client/server or Web-based.
g. Language type: 3GL, 4GL, or application generators used in development.
h. Overall data quality rating assigned by the ISBSG: A, B, C or D indicating very good to unreliable.
i. Function points data quality rating assigned by the ISBSG: A, B, C or D ranging from very good to unreliable.
2.2. Data Preprocessing
A set of criteria was defined to ensure data quality, relevance to current industry needs, suitability to the testing context and adequacy for statistical analysis, as follows:
1) Data Quality
a. ISBSG quality rating:
Data quality ratings of A and B were selected to reduce risk and improve confidence in the results.
b. Function point size quality:
When IFPUG function points were used for the measurement of size, only the un-adjusted function point value was considered. Function point data quality ratings of C and D were excluded from the data.
2) Data Relevance
ISBSG data consist of projects reported since the early 90s. Data prior to 2000 and projects with an architecture type of "standalone" were removed while client/server or Web-based projects were considered for modelling.
3) Data Suitability
To exclude trivial projects, the following filters were applied:
a. Total normalized work effort (full life cycle effort for project) equal to or greater than 80 hours.
b. Efforts reported for testing greater than or equal to 16 hours.
c. Types of testing other than functional testing were excluded.
4) Data Adequacy
a. Application group chosen: business.
b. Development type chosen: new development and re-development.
2.3. Generation of Datasets
Applying the filters related to the criteria for data selection and removal of outliers resulted in 142 data points, which were then grouped to form four datasets:
Dataset A: This dataset consists of all 142 data points including project functional size measures reported in IFPUG 4.1 or COSMIC FP. For this study, they were not differentiated within dataset A as they correlate well even though the relationship is not the same across all size ranges [9] .
Dataset B: In the case of dataset A, projects with an architecture field value of "standalone" were eliminated from the original ISBSG data set, while "blanks" were retained. To be very specific about the architecture type, "blanks" were also eliminated from dataset A to arrive at dataset B, with 72 data points.
Dataset C: Data set C is made up of projects where functional size was reported in COSMIC function points. It is a subset of data set A and has 82 data points.
Dataset D: Dataset D includes only projects where functional size was reported in IFPUG function points. It is another subset of dataset A and contains 60 data points.
3. Data Analysis
3.1. Strategy
The following strategy was adopted for data analysis:
a) Identify data point subsets exhibiting different levels of testing productivity.
b) Analyze these subsets to identify the possible causes for the differences in productivity.
3.2. Identification of Test Productivity Levels
The scatter diagram in Figure 1 depicts a large dispersion between functional size and test effort, the independent and dependent variables, respectively. The pattern is closer to wedge-shaped and is typical of data from large repositories [10] .
Within the dataset of Figure 1, there are candidate groups exhibiting both large economies of scale and large diseconomies of scale. The rate of increase of test effort is not the same for all similar functional sizes. Analyzing various slices of data brought out different testing productivity levels (Figure 2).
As economies and diseconomies of scale correspond to different productivity levels, a new term "test delivery rate" (TDR) was defined to describe project testing productivity. TDR is the rate at which software functionality is tested as a factor of the effort required, and is expressed as hours per functional size unit
Figure 1. Scatter diagram: size versus test effort.
Figure 2. Multiple data groups representing different economies of scale.
(hr./FSU). Functional size unit (FSU) refers to either IFPUG or COSMIC function points, depending upon the sizing method used for measurement. The four varying levels of productivity are referred as "TDR levels". TDR being the effect, the characteristics of the projects falling into each level were then investigated to identify the underlying causes. Due to the highly dispersed nature of TDR level 4, only TDR levels 1 to 3 were taken up for further analysis and development of the estimation models.
3.3. Identification of Candidate Characteristics of Projects
Previous research work [11] [12] based on data from hundreds of software projects has indicated that team size and schedule (duration of the project) within a particular domain affect the productivity of development projects. As software testing is one of the phases of development, project attributes such as domain, team size and elapsed time are likely causes for test productivity, too. Testability of the software components, i.e., quality of the software delivered for testing, is critical for reducing the testing cost [13] and hence the effort for testing. The quality of the software delivered for testing can be determined by the extent of verification and validation activities carried out during the development process.
From these previous studies, therefore, in choosing candidates of interest for our investigations we selected the following project characteristics:
Team size: We classified team size into three categories typically present in the industry: (a) 1 - 4 persons, (b) 5 - 8 persons and (c) more than 8 persons resenting small, medium and large team sizes, respectively.
Elapsed Time: Elapsed time in calendar months was derived from the "project elapsed time". Based on this attribute, projects were classified into three groups: (a) 1 - 3 months, (b) 4 - 6 months and (c) greater than 6 months, referred to as small, medium, and large, respectively.
V & V rigor: This attribute was derived from the data fields related to the "Documents & Techniques" category in ISBSG, which indicates the degree of rigor applied during verification and validation. Two ratings are proposed for V&V rigor (Table 1).
Application domain: This attribute was derived from the ISBSG data field "Industry Sector". Considering the number of data points available for different industry sectors, the application domain was classified into three categories, namely: (a) banking, financial services and insurance (BFSI) (b) education and (c) government (govt).
Table 1. V&V rigor rating scheme.
The group of projects contributing to each TDR level was termed the project group. Accordingly, project group 1 (PG1), project group 2 (PG2) and project group 3 (PG 3) refer to TDR levels 1, 2 and 3 discussed in Section 3.2. Based on the percentage of projects falling into each of the project groups for the four attributes of interest (Table 2), we were able to characterize the project groups.
Close to half of the BFSI projects (46%) fell into PG3 followed by a third in PG2. All education projects fell into PG1 while slightly more than half of the government projects fell in PG2. Close to two thirds of projects with a small team size fell into PG1, while 82% (46% + 36%) those with a medium team size were distributed between PG1 and PG2. Slightly less than 50% of the large team size projects fell into PG3. Close to two thirds of projects with a small elapsed time went into PG1, while 71% (38% + 33%) of those with a medium elapsed time were spread between PG2 and PG3. Similarly, PG2 and PG3 shared 72% (40% + 32%) of the projects with a large elapsed time. Projects with higher V&V rigor had a two thirds presence in PG1 while 77% (38% + 39%) of the lower V&V rigor projects were divided more or less evenly between PG2 and PG3.
The above analysis reveals that the three project groups have certain distinctions with respect to the domain, team size, elapsed time and V&V rigor besides test productivity. To establish statistical significance, a test of hypothesis was performed and the p value computed. The significance test conducted on the three project groups with respect to these attributes resulted in a p value of less than 0.001 for V&V rigor and domain and less than 0.1 for team size and elapsed time. This further establishes that the variations across the three project groups are reasonably significant and the attributes identified are potential contributors to test productivity.
Table 2. Analysis of project characteristics―Dataset A.
The results of the analysis of the project characteristics in the three datasets A to C, excluding dataset D, demonstrate similar behavior. Characteristics of project groups PG1 to PG3 based on these attributes are summarized in Table 3.
3.4. Identification of the Independent Variables
1) Size
It has been observed that functional size is the most accepted approach for measuring size, as sensitivity to changes in functional size has a greater impact on project effort [14] [15] . Here, correlation coefficients computed using dataset A, between size and test effort values of 0.9035 for PG1, 0.8572 for PG2 and 0.8572 for PG3, indicate good correlation of functional size with effort. Size was therefore chosen as the primary independent variable.
2) Non-Size Variables
Size being the main independent variable, other independent variables were next examined for significance of incorporating them into estimation models. It has been observed [13] that "testability of software components", meaning the quality of the software delivered for testing and testing processes followed while testing, are critical factors for reducing testing effort and improving software quality. To accommodate these process factors two new variables representing development process quality and testing process quality were defined and investigated as follows.
a) Development Process Quality Rating (DevQ)
The process followed during development was rated by considering the nature of the development life cycle followed and the artefacts produced, based on the following project attributes:
Ÿ Standards followed.
Ÿ Distinct development life cycle phases followed.
Ÿ Verification activities carried out during development.
The ISBSG data field "software process" has one of the values―CMMI, ISO, SPICE, PSP or any such standard followed during development. A set of fields representing "Documents and Techniques" exists in the ISBSG data providing information on the life cycle phases adopted and verification activities carried out during development. Based on these, a rating for DevQ was developed, as shown in Table 4.
b) Test Process Quality Rating (TestQ)
While reviewing, the data related to the testing process followed, it was found
Table 3. Characteristics of project groups.
Table 4. Rating for development process (DevQ).
that there were not enough fields in the ISBSG data to capture the details of the testing process, such as testing techniques adopted, levels of testing executed, test artefacts produced, reviews of test cases etc., to gauge the extent of testing. This notwithstanding, it was possible to classify the test process rating broadly into two categories (Table 5).
3.5. Analysis of DevQ and TestQ
Projects in data set A were analyzed in terms of DevQ and TestQ:
Ÿ 36%, 49%, and 15% of the projects were found to be in DevQ with ratings 0, 1 and 2, respectively.
Ÿ 80% and 20% of the projects had TestQ ratings 0 and 1 respectively.
To further justify the inclusion of these variables, two statistical tests were carried out to quantify their significance (Table 6):
Ÿ the Kruskal-Wallis Test for DevQ as it involved three categories, and
Ÿ the Mann Whitney Test was applied for TestQ.
The p value indicated that size, DevQ and TestQ were statistically significant.
4. Estimation Models
4.1. Portfolio of Models
The linear form of relationship between input and output variables was chosen to build models for effort estimation. Linear regression analysis, a well-known and well understood algorithm in statistics and machine learning, does not require much training data, and is easily interpreted by project managers. Parametric models are objective, repeatable, fast and easy to use, and can be used early in the life cycle if they are properly calibrated and validated [16] . A set of 24 models under four portfolios were generated (Table 7) using datasets A to D.
Portfolio A models based on dataset A:
Models 1, 2 and 3 are for each project group using size as the independent variable.
Models 4, 5 and 6 use both size and DevQ as independent variables and relate to project groups 1, 2 and 3 respectively.
Models 7, 8 and 9 use size, DevQ and TestQ as independent variables and represent project groups 1, 2 and 3 respectively.
Portfolio B models based on dataset B:
Models 10, 11 and 12 relate to project groups 1, 2 and 3 respectively using size as independent variable.
Table 5. Rating for test process (TestQ).
Table 6. Test of significance for independent variables.
Table 7. Portfolio of estimation models.
Models 13, 14 and 15 belong to project groups 1, 2 and 3 respectively using size and DevQ as independent variables.
Models 16, 17 and 18 refer to project groups 1, 2 and 3 using size, DevQ and TestQ as independent variables.
Portfolio C models based on dataset C (COSMIC FP Projects):
Model 19, 20 and 21 relate to project groups 1, 2 and 3 respectively using size as independent variable.
Portfolio D models based on dataset D (IFPUG FP Projects):
Model 22, 23 and 24 relate to project groups 1, 2 and 3 respectively using size as independent variable. Depending upon the number of independent variables, model equations have coefficients A, B, D1, D2, T1 and T2 (Table 7), which were used to estimate the value for test effort for specific values of size, DevQ and TestQ as explained next.
Using estimation models based on size:
Test effort for a particular functional size can be estimated from models using the following equation representing size based estimation models:
T e s t E f f o r t = A + ( B × ( S i z e ) ) (1)
Test effort for a particular functional size can be computed by using the values of A and B from Table 7 and substituting functional size for "size" in Equation (1).
Using estimation models based on size and DevQ:
Test effort for a particular value of functional size and DevQ can be estimated from models using the following equation:
T e s t E f f o r t = A + ( B × S i z e ) + D 1 + ( D 2 × S i z e ) (2)
Test effort for a particular functional size where rating for DevQ is available can be computed using Equation (2). D1 and D2 have different values based on the value of DevQ. Appropriate values from Table 7 are to be chosen depending on whether DevQ = 0 or Dev Q = 1. For DevQ = 2, the value is 0, the base value considered while modelling.
Using estimation models based on size, DevQ and TestQ:
The equation for estimating Test Effort for particular values of size, DevQ and TestQ from the model has the form:
T e s t E f f o r t = A + ( B × S i z e ) + D 1 + ( D 2 × S i z e ) + T 1 + ( T 2 × S i z e ) (3)
Equation (3) can be used for computing test effort estimate for a particular functional size when ratings for both DevQ and TestQ are available. Values for D1 and D2 are to be chosen from Table 7 depending upon the input value of DevQ, is either 0 or 1. Values for T1 and T2 are provided for TestQ = 0. Values for DevQ = 2 and Test Q = 1 are zero, as they were the baseline for the modelling.
An estimator chooses the project group by mapping the characteristics of the project to be estimated to the attributes of project group and selects the related data set in order to choose the closest model for estimation.
4.2. Evaluation of Estimation Models
The quality of estimation models was evaluated using criteria such as coefficient of determination (R2), Adj R2, magnitude of relative error (MRE), median magnitude of relative error (MedMRE) [10] (Table 8).
The value of R2 for portfolio A ranged between 0.74 and 0.86, and that of Adj R2 ranged between 0.73 and 0.83 indicating a strong relationship between the independent variables-size, DevQ and TestQ with the dependent variable test effort in all models.
The value of MedMRE ranging between 0.22 and 0.28 shows that the error levels between the estimate and actual are within the range of 22% to 28% for 50% or less of the samples, which is practical considering the multi-organiza- tional data used for building the models.
Similar observations can be made for rest of the models.
Table 8. Quality of estimation models.
4.3. Comparison of Model Performance
4.3.1. Predictive Performance of Models
The criterion used to evaluate the predictive quality of an estimation model was PRED (l) = k/n, where k is the number of projects in a specific sample of size n for which MRE <= l. In the software engineering literature, an estimation model is considered good when PRED (0.25) = 0.75 [17] or PRED (0.30) = 0.70 and PRED (0.20) = 0.80 [10] . PRED (0.25) = 0.75 means 75% of the samples should have MRE values less than or equal to 0.25. While an MRE error level in 75% of the population less than 0.25 is the expectation of this criterion, multi-organiza- tional data such as in ISBSG data exhibit large MRE for 75% of the population.
To compare the performance of models, MRE values for 50% and 25% of the population in addition to 75% were taken into consideration. Each vertical bar in the charts (Figures 3-7) depicts the MRE value for 50% in the middle with either extremes showing MRE values for 25% and 75% of the population for each of the identified model. The middle points of each bar (MRE for 50% of population) are connected by a line to visualize the difference between successive models. This point is referred simply as MRE in the following discussions and was used to compare the performance of the models.
Figure 3. Performance of data set A models.
Figure 4. Performance of size based models.
Figure 5. Size & DevQ models in data set A and B.
Figure 6. Size, DevQ & TestQ models in data set A and B.
Figure 7. Performance of COSMIC vs. IFPUG models.
4.3.2. Dataset A Models in Portfolio A
There are 9 models in portfolio A. A comparison of these nine models reveals how predictability varies between project groups and while using different independent variables. Figure 3 depicts the MRE levels of models corresponding to PG1 (the leftmost three bars), PG2 (the next three bars) and PG3 (the rightmost three bars).
Within PG1, the model with size, DevQ and TestQ as independent variables (model 7) demonstrate lower MRE for 50% of the population compared to the model with size, Dev Q (model 4) which is lower than the model with size alone (model 1).
A similar pattern is observed in PG3 for models 3, 6 and 9. In the case of PG2, the model using size, DevQ and TestQ (model 5) exhibits higher MRE compared to other models in PG2 (models 2 & 8) as well as for all the models in dataset A.
4.3.3. Size-Based Models
A comparison of all models using only size as an independent variable across all portfolios shows under which context size-based models provide better predictability. Figure 4 illustrates size-based models from each portfolio, the first three bars corresponding to each project group in portfolio A, the next three corresponding to each project group in portfolio B and so on.
Ÿ Size-based models for project groups PG1 and PG3 are better than PG2 with the exception of portfolio D.
Ÿ PG3 size models, in general perform better than PG1 and PG2 except for the last model (Model ID 24).
4.3.4. Models in Portfolios A and B
Portfolio B models were developed using a subset of data used for portfolio A. Portfolio B models were more specific to web or client/server architecture, unlike portfolio A models where there was an approximation due to differences in architecture. A comparison between the models across portfolios A and B using the independent variables DevQ and TestQ along with size helped to make certain observations. Figure 5 depicts size & DevQ models for PG1, PG2 and PG3 for dataset A and B, while Figure 6 illustrates size, DevQ and TestQ models for PG1, PG2 and PG3 for data set A and B.
Examination of Figure 5 reveals that models in portfolio B (models 13, 14,15,16 & 18) performed much better than models in portfolio A (models 4 to 9) with model 17 being an exception.
4.3.5. COSMIC and IFPUG Models
The performance of COSMIC (dataset C) and IFPUG (dataset D) models was compared next using size-based models from portfolio A as the reference. Both COSMIC and IFPUG data are subsets of dataset A consisting of projects measured using the corresponding sizing method. This comparison can help to evaluate prediction accuracy of COSMIC-based models versus IFPUG-based models. Figure 7 depicts PG1 models for dataset A (model 1), COSMIC (model 19), IFPUG (model 22), PG2 models for data set A (model 2), COSMIC (model 20), IFPUG (model 23) and PG3 (model 3), COSMIC (model 21) and IFPUG (model 24).
COSMIC-based estimation models using dataset C had better performance than IFPUG-based estimation models using data set D, with the exception of PG2 (model 20). COSMIC-based PG3 model demonstrated the best predictability. Furthermore, the R2 values for COSMIC-based models ranged from 0.73 to 0.87 while that of IFPUG-based models ranged from 0.70 to 0.78 (Table 8). The MedMRE value for COSMIC-based models ranged between 0.19 and 0.30 compared to IFPUG based models ranging between 0.25 and 0.33 (Table 8) demonstrating better accuracy of COSMIC based models.
This research work explored software testing from the perspective of estimation of efforts for functional testing. The ISBSG database, with its wealth of project data from around the globe, was used for the first time in building effort models for functional testing. The analysis of the data revealed three test productivity patterns representing economies and diseconomies of scale, based on which characteristics of the corresponding projects were investigated. Three project groups, characterized by domain, team size, elapsed time and rigor of verification and validation, and related to three productivity patterns were found to be statistically significant. Within each project group, the variations in test effort could be explained, apart from the functional size, by 1) the processes executed during the development, and 2) the processes adopted for testing.
Two new independent variables, DevQ and TestQ were identified as influential in the estimation of effort. A total of 24 models were built, using combinations of the three independent variables. The quality of each model was evaluated using established criteria such as R2, Adj R2, MRE and MedMRE. As these models were built from ISBSG data, they could serve as an industry benchmark for functional test efforts. Test estimation models using projects measured in COSMIC function point exhibited better quality and resulted in more accurate estimates compared to projects measured in IFPUG function points.
The models are applicable only for the ranges of size in the data set and for testing of business applications. The models generated are not applicable for enhancement projects. These limitations can be overcome by generating specific models for enhancements or real-time projects, using an approach like the one followed in this work. This may require identification of additional project characteristics, as well as other variables influencing testing effort. PG4―the fourth group of project data points remains to be analyzed.
The process factors used for rating DevQ and TestQ can be further refined within organizational context. There could be other variables that influence test efforts in specific contexts, which would require further study and analysis. The estimation models designed can be further refined by considering testing techniques adopted as a parameter to evaluate their impact and then used to build estimation models.
The authors would like to acknowledge the support provided by COSMIC consultant Srikanth Arvamudhan and statistician Sriram Ramachandran during this work.
Jayakumar, K.R. and Abran, A. (2017) Estimation Models for Software Functional Test Effort. Journal of Software Engineering and Applications, 10, 338-353. https://doi.org/10.4236/jsea.2017.104020
1. ISO/IEC 20926:2009 (2009) Software and Systems Engineering—Software Measurement—IFPUG Functional Size Measurement Method. International Organization for Standardization (ISO), Geneva.
2. Jayakumar, K.R. and Abran, A. (2013) A Survey of Software Test Estimation Techniques. Journal of Software Engineering and Applications, 6, 47-52. https://doi.org/10.4236/jsea.2013.610A006
3. Abran, A. (2010) Software Metrics and Software Metrology. Wiley & IEEE Computer Society Press, New Jersey. https://doi.org/10.1002/9780470606834
4. ISO/IEC 19761 (2011) Software Engineering—COSMIC—A Functional Size Measurement Method. International Organization for Standardization (ISO), Geneva.
5. COSMIC (2015) The COSMIC Functional Size Measurement Method, Version 4.0.1, Measurement Manual, The COSMIC Implementation Guide for ISO/IEC 19761:2011. Common Software Measurements International Consortium, Canada, April 2015. http://www.cosmic-sizing.org
6. ISBSG (2013) Repository Data Release 12—Field Descriptions, "e.Field Descriptions —Data Release 12. Pdf" Document Provided as a Part of Data Set. International Software Benchmarking and Standards Group.
7. Bala, A. (2013) Impact Analysis of a Multiple Imputation Technique for Handling Missing Value in the ISBSG Repository of Software Project. PhD Thesis, Ecole de technologie superieure, University of Quebec, Montreal (Canada), 17 October.
8. Bala, A. and Abran, A. (2016) Use of the Multiple Imputation Strategy to Deal with Missing Data in the ISBSG Repository. Journal of Information Technology & Software Engineering, 6, 171.
9. Dumke, R. and Abran, A., Eds., (2011) COSMIC Function Points, Theory and Advanced Practices, Chapter 3.5: Measurement Convertibility—From Function Points to COSMIC FFP. CRC Press, New York.
10. Abran, A. (2015) Software Project Estimation: The Fundamentals for Providing High Quality Information to Decision Makers. John Wiley & Sons & IEEE Computer Society, New Jersey. https://doi.org/10.1002/9781118959312
11. Putnam, D. (2005) Team Size Can Be the Key to a Successful Software Project. Quantitative Software Management Inc., USA. http://www.qsm.com
12. Armel, K. (2012) Top Performing Projects Use Small Teams Deliver Lower Cost, Higher Quality, Blog Posting. Quantitative Software Management, Inc., USA.
13. Taipale, O. (2007) Observations on Software Testing Practice. PhD Thesis, Lappeenranta University of Technology, Finland.
14. Bharadwaj, M. and Rana, A. (2015) Estimation of Testing and Rework Efforts for software Development Projects. Asian Journal of Computer Science and Information Technology, 5.
15. Hill, P., ISBSG (2010) Practical Software Project Estimation: A Toolkit for Estimating Software Development Effort and Duration. McGraw-Hill, New York.
16. Galorath, D. (2015) Why Can't People Estimate: Estimation Bias and Mitigation. Conference presentation at IT Confidence Conference, Rio de Janerio, October 2015.
17. Conte, S.D., Dunsmore, D.E. and Shen, V.Y. (1986) Software Engineering Metrics and Models. The Benjamin/Cummings Publishing Company, Inc., Menlo Park.
●JSEA Subscription
●Most popular papers in JSEA
●About JSEA News


Prem Singh replied to Dr. Krishna Kumari Challa's discussion What do you do to stand out as an artist?
"Art is mainly a quest for one's own truth. It has nothing to do with vanity or self glorification"
"Hi Trina, Pleased to meet u. Best wishes"
"Thank u for the welcome. Wishing u all the best."
"Welcome to art lab. Glad to see you here. Now that you are in a lab, you know what to do. There is a critics corner here. All the best Krishna"


text:
Healthy Travel Tips »
Healthy Travel Tips » Stay safe on vacation
Best National Parks in Canada
Canada's geography is grand in both scale and scenery, giving locals and visitors enough adventures to last a lifetime. Much of Canada's brilliant landscape can conveniently be found in the confines of its incredible national parks, offering the opportunity to get up close and personal with natural wonders ranging from glaciers to granite peaks. To determine the best of the best, U.S. News considered scenic beauty, accessibility, expert opinions and traveler sentiment to determine the best national parks in Canada. Have a favorite? Vote below to influence next year's ranking. (Note: Some of the below destinations may be affected by the coronavirus outbreak. Check with the CDC and the U.S. Department of State before traveling.)
Read Best Vacations Methodology »
Kluane National Park and Reserve
Auyuittuq National Park
Cape Breton Highlands National Park
Gwaii Haanas National Park Reserve
Grasslands National Park
Point Pelee National Park
Belongs on List?
#1 in Best National Parks in Canada
Jasper National Park is by far the best place to experience the Rocky Mountains. The largest national park in the Canadian Rockies offers endless opportunities to go hiking, biking, boating and more. Jasper also holds the unique distinction of being the world's second-largest dark sky preserve, so you can expect excellent conditions for stargazing when you visit. To soak up the best of this Alberta national park, drive down the famous Icefields Parkway, which is considered one of the most scenic drives in the world.
Best Hotels | Flights | Things to Do
Gros Morne National Park's geography is so special, it was declared a UNESCO World Heritage Site. Situated on the island of Newfoundland, the park stands out for its striking fjords and the Tablelands, where visitors can walk upon the exposed Earth's mantle. Gros Morne's location on the Gulf of St. Lawrence also affords access to impressive coastal scenery, which is best viewed along the spectacular Green Gardens trail. Meanwhile, ambitious hikers can climb to Gros Morne Mountain's summit on the trail of the same name.
If you live for hiking, then Kluane National Park and Reserve needs to be on your bucket list. The park features 23 different trails and routes that range from quick half-hour hikes to multiday treks. Kluane is also home to 17 of Canada's 20 highest mountains, including the highest peak in the country, Mount Logan. Whether it's hiking, rafting next to glaciers on the Alsek River or enjoying a "flightseeing" excursion (or air tour) over Canada's largest icefield, there's an adventure activity that'll appeal to you here.
The word "Yoho" is an exclamation of awe in the First Nations Cree language, and you'll certainly be amazed by this park in British Columbia's Rocky Mountains. Highlights include Takakkaw Falls (one of the highest waterfalls in Canada) and the aptly named Emerald Lake (the park's largest lake). You'll also want to save time for checking out the Burgess Shale fossils, which are estimated to be more than 500 million years old.
Though it's the smallest national park in the Canadian Rockies, Waterton Lakes National Park manages to pack a lot of impressive sights within its 124,800 acres. Situated in an unassuming corner of southwestern Alberta, Waterton Lakes borders Montana's Glacier National Park and offers visitors jaw-dropping views of waterfalls, mountains, wildflowers and wildlife. Can't-miss sights include the Chief Mountain Overlook (where the park's mountains seamlessly transition into the prairie), the Waterton-Glacier International Peace Park (a UNESCO World Heritage Site) and the paved Red Rock Parkway (which promises outstanding views of the park's highest peak, Mount Blakiston).
Located within the Arctic Circle is Auyuittuq National Park, one of Canada's wildest national parks. The park is not easily accessible; you have to fly to remote Nunavut before taking another plane to one of two Inuit villages, where adventure outfitters guide you through sky-high fjords to get into the park. But once inside, you'll find breathtaking landscapes composed of granite peaks and glaciated valleys, as well as arctic wildlife, including polar bears and caribou.
Situated on the northern tip of Nova Scotia, Cape Breton Highlands National Park is often called the area "where the mountains meet the sea" by Canadians. The park's greatest attribute, the Cabot Trail, is the best way to take in this varied scenery. The paved road winds along towering coastal cliffs and through lush forestry that blankets the park. For an unforgettable hike, hit up the oceanfront Skyline Trail at sunset.
Gwaii Haanas National Park Reserve's remote location on Moresby Island off of the western coast of Canada is no doubt a challenge to reach, but once you get an eyeful of its incredible offerings, you'll realize the destination is worth the journey. Gwaii Haanas features moss-laden forests, dramatic coastal terrain and even UNESCO-sanctioned ruins. Keep in mind that there are no marked trails at this park; everything must be navigated the old-fashioned way (with a map and compass).
Nestled in the wooded highlands of Manitoba, Riding Mountain National Park beckons to wildlife lovers and hikers alike. As you traverse the park's more than 250 miles of hiking trails, keep your eyes peeled for wolves, lynxes, black bears and moose, among other animals. When your feet need a break, head out on the water: RMNP encompasses more than 1,900 lakes. You can canoe, kayak or hop in a motorized boat to explore the aptly named Clear Lake, which is considered the clearest lake in the southern prairies.
#10 in Best National Parks in Canada
This Saskatchewan park is Canada's first and only park to preserve and represent the mixed-grass prairie ecosystem. The ecosystem is home to animals like bison, black-tailed prairie dogs and short-horned lizards, and visitors can even see dinosaur fossils here. To see these animals and the park's terrain, plan a road trip along the Ecotour Scenic Drive at dawn or dusk. Grasslands is also the darkest dark sky preserve in Canada, which means stargazers are in for some spectacular celestial views come nighttime.
Point Pelee National Park is a bird-watcher's paradise: More than 390 different species have been sighted here. Each May, the Ontario park hosts the Festival of Birds so visitors and birders can watch the spring migration and participate in a variety of programs and classes about birds. Plus, travelers can take a shuttle provided by the Friends of Point Pelee to the southernmost tip of mainland Canada, where they can watch the migrations of birds in spring and dragonflies, monarch butterflies and birds in fall. The park is also home to 12 miles of sandy beaches.
Vote to Add these Destinations to the Rankings
Pacific Rim National Park Reserve
Fundy National Park
Kejimkujik National Park and National Historic Site
Elk Island National Park
Prince Albert National Park
Wapusk National Park
Best U.S. National Parks
Best National Parks in the World
Best Adventure Vacations in the U.S.
Best Places to Visit in Canada

redpajama_set_name: RedPajamaCommonCrawl


This recipe takes on the full flavor profiles of Thailand at its finest. It starts with a broth that pairs with chili, garlic, lemon grass, turmeric and ginger that all go in harmony with the coconut milk. This recipe incorporates Sizzlefish peeled and deveined shrimp and sea scallops for an extra protein punch. Try it out for a warm, cozy night that is sure to leave your tastebuds wanting more!
Next, in a large pot, bring to boil your herbs, spices, and broth. Let is simmer about 10 minutes then add in your cabbage and simmer another 5 minutes.
Next add in your coconut milk and cooked shrimp or scallops.
Mix in the your peppers (chiles) keep on low until ready to serve.
add cilantro and serve with sirirachi on top.


The Fitzwilliam Quartet made a thoroughly admirable recording of the fifteen string quartets of Shostakovich back in the 1970s and, until now, I have lived happily with them. But a recent acquisition of the quartets played by the Beethoven Quartet has completely overturned my loyalties. From 1938 onwards the Beethovens worked closely with Shostakovich, and gave the first performance of 13 of the 15 quartets. In particular, the first violin – Dmitri Tsyganov, the viola – Vadim Borisovsky and the cellist – Sergei Shirinsky come over as passionate solo players within the quartet. The second violin was Vasily Shirinksy, and I think that passion defines the quality I love in these performances.
The transfers from the Melodya LPs of the 1950s and 60s are very well done by Doremi (at least, for those quartets I have listened to so far). I love these string quartets, but I'm afraid the Fitzwilliams go back on the shelf, and the Beethovens stay very near at hand.


The full details of Apple's second-generation iPad Air will be revealed in just four short days, but Apple.Club.tw has published a photo of an alleged iPad Air 2 logic board that reveals Cupertino has a souped up A8X chip ready for the new tablet.
If the logic board is genuine, it confirms rumors that Apple will improve the Air's performance with a special A-chip variant, like the A5X and A6X processors used in previous iPads. The logic board also appears to contain 2GB of RAM, though the chip numbers are a bit hard to read.
The characters on the Elpida chip are blurry, but it looks like the second character – used to indicate the amount of RAM – is an 'A', which stands for 2GB, while an '8' would be used to represent 1GB.
Apple is hosting a town hall press event at its Cupertino campus on Thursday, October 16th at 10AM PT. Invites to the 'It's Been Way Too Long' event didn't leave any clues as to what Apple plans to reveal, but it's expected to announce new iPads and a Retina iMac to go with the public release of OS X Yosemite.
Other new iPad Air 2 features may include an anti-reflective coating, slimmer design, Touch ID, Apple Pay, recessed volume buttons, refined speaker grills, and a gold color options.


text:
https://www.gameinformer.com/funny-to-a-point/2019/01/18/the-second-annual-not-50-awards
The Second Annual Not 50 Awards
by Jeff Marchiafava on Jan 18, 2019 at 05:31 PM
Along with holiday parties, festively fugly sweaters, and tons of treats I totally shouldn't be eating, the end of the year always brings with it a sobering realization: I work with a bunch of crazy people. That's the only logical – and frankly generous – conclusion I can reach every year as I listen to my coworkers spout obviously wrong opinions for DAYS on end during the blitzkrieg of meetings that decide our Top 50 list. Sometimes the arguments for or against any given game are so absurd that I feel like a robot trying to process a logical paradox – all I can do is scream "DOES NOT COMPUTE!" over and over as I run out of the meeting room with sparks shooting from my ears. That's not a joke, people – it literally happened during last year's Breath of the Wild debate!
Luckily, I've discovered a sanity-saving workaround to the dilemma – I simply disavow the results of our year-end list completely and make my own damn awards instead. I am not bragging when I say they are without a doubt the best awards ever, because not only do I unilaterally pick all the winners (so you know they're right!), but I also make up the categories too! So, without further ado, I present to you the second annual Not 50 Awards!
…and yes, I know "Not 50 Awards" would've been a better name for my subversive take on E3's Hot 50 awards, but I already named those The Dubys, so just roll with it, okay?
Desert Island Award: Tetris Effect
I feel sorry for people who don't get Tetris. To them, it's just a mindless puzzle game where you endlessly drop dumb shapes into a well for no reason, and yet idiots won't stop gushing about it being the best game of all-time. Explaining to them how Tetris Effect takes that perfect experience and somehow makes it even perfecter would be impossible. Basically, it adds crazy visuals and music that syncs to your movements. See?! If you don't get it, you just don't get it!
Thankfully, I do get it, and I could play Tetris Effect FOREVER. So congratulations, Monstars, you made an amazing Tetris game! Not like that's a hard thing to do. Well, unless you're Ubisoft…or EA…or THQ – wait a minute, how do so many people screw up Tetris?!
Best Progress Derailer Award: Mahjong (Yakuza 6)
Do you consider yourself a connoisseur of story-driven games? Well, I got news for you: All the BioWare and CD Projekt Red and David Cage games in the world ain't got sh-- on Yakuza. By my estimate, Yakuza 6 has roughly one BILLION hours of story cutscenes, each immaculately choreographed and voice-acted. Seriously, you can't walk five feet in Kamurocho without triggering some ridiculous, character-building cutscene with a random stranger (usually a pervert) that ties into the Yazuka lore Sega has been building on for 15 years.
All that wouldn't mean much if the story isn't good, but guess what? It is…probably? The truth is I don't know, because as soon as I opened up the Mahjong mini-game in Yakuza 6, I became hopelessly addicted. I know there's a little girl with an even littler baby that I should be rescuing or something, but I can't pry myself away from the Mahjong tables – the only time I leave the parlor is to shake down random thugs for more betting money, because I still don't fully understand the rules of Mahjong. Actually, is it too late to change my Desert Island award?
Best Catalyst For Self-Reflection: Horse Rides (Red Dead Redemption II)
Last year, no criticism of a video game broke my brain more than the widespread and oft-repeated argument lodged against Red Dead Redemption II: "The horse rides take too long! Wahhhhh!!!"
Oh, I'm sorry, you mean the horse rides through the most meticulously detailed and positively gorgeous open world ever created in the history of video games? In a game where escaping the trappings of modern society and finding your own path in a dangerous, changing world is half the effing point? You're telling me that we all waited eight long years for Rockstar to craft its most ambitious and immersive game yet, and your main complaint is you can't speed through it fast enough?!
If Red Dead's horse rides are detracting from your enjoyment of the game, here's a humble suggestion: Use that time to ask yourself what in the holy hell you want from a video game! To dash from one objective to the next as quickly as possible while the designer pats you on the head for being an efficient gamer? Or to lose yourself in a world and story and role that you can't experience anywhere else?
Besides, it's not like the horse rides are super demanding anyway – if you turn on cinematic mode, the horse will even steer for you! Not well, mind you, but it's something!
Never Wanna Hear About It Again Award: Dragon Ball FighterZ
Look, I'm thrilled Dragon Ball FighterZ turned out to be a great game, and I'm even more thrilled that Dragon Ball fans are thrilled that it's a great game – everything is very thrilling and great! I just don't ever want to hear another word about it ever again. It was like three straight years of build up and character reveals and trailers – it's too much for non-Dragon Ball fans!
To be fair, I kinda did this to myself when I voluntarily analyzed every Frieza'n character in the game, and I've been trying to push all of that arcane knowledge out of my brain ever since. For Gotenks' sake, please don't make a sequel!
Undeserved Genius Award: Into The Breach
This award may sound like an insult, but it really isn't. Despite its giant-mechs-smashing-equally-giant-bugs-to-gooey-bits premise, Into The Breach is one of the smartest, thinkiest strategy games I've ever played. It's like four-dimensional chess, only with buttloads more missiles. To be honest, it's way too smart for me, but it still makes me feel like a genius every time I successfully clear a level without taking any casualties. Hence, the Undeserved Genius award. In fact, creating a game that makes non-geniuses feel like geniuses is probably a sign that you are in fact a genius, which in Subset Games' case is praise that IS deserved. Good lord, I feel even dumber after writing this – just go play the game already!
Forget The World And Blow Sh-- Up Award: Just Cause 4
I guess this the only logical follow-up to the last entry, but I really don't have anything else to add; you can already watch 25 absurd videos that outline my intensely stupid love affair with Just Cause 4.
Low Expectations Award: Detroit: Become Human
You may have noticed that Detroit: Become Human was included in G.I.'s Top 50 list. You may have also noticed that Detroit: Become Human did not garner a single moment on our Top 10 Moments list, or a single character on our Top 10 Characters list – in fact, it wasn't even brought up as a contender in those categories. If you're wondering how a purely story-driven game can make the Top 50 without having a single standout moment or character worth championing, SO AM I.
Instead, the arguments in Detroit's favor were couched in language like, "I mean yeah, the story is totally overwrought, but it's got some interesting ideas!" and "Sure, the racial allegories were heavy-handed, but they weren't as bad as Beyond: Two Souls' Najavo sequence…" Is this really all it takes to make David Cage fans happy? What kind of Stockholm Syndrome are we dealing with?!
On the bright side, one of Detroit's protagonists handily earned an entry on our Top 10 Dorks list, which is an award I can totally stand behind. Who says I'm not a team player?
Really Gotta Play More Award: Monster Hunter: World
First let's dispense with the formalities: Yes, I named my cat "Kat" in Monster Hunter – what're you gonna do about it?!
Anyway, I played a bunch of Monster Hunter: World when it first came out, but then Joe transformed me into a Bohemian peasant with the wave of his review wand, and I never quite got back to the beast-slaying action. It's been riding high on my pile of shame ever since, earning it this year's Really Gotta Play More Award. On a side note, this was the Not 50's most competitive category, with close runners-up including Assassin's Creed Odyssey, Red Dead Redemption II, Yakuza 6 – you know what, this is a lot more depressing than I anticipated, so let's just move on…
Damn Fine Game Award: Spider-Man
Poor, poor Spider-Man. If it weren't for that darn Kratos, Insomniac's open-world webslinger would've easily nabbed G.I.'s Best Sony Exclusive award this year. And let's face it – it could win the Best Microsoft Exclusive award pretty much ANY year. I originally wrote that as a joke, but then decided to test it out by looking up our old awards the year Insomniac released Sunset Overdrive, and sure enough, it won the Microsoft award – and that game was basically Spider-Man without the webs!
The fact that Spider-Man got completely overshadowed in G.I.'s awards is a testament to how strong Sony's line-up is, and what a great year for games it was in general. I'm guessing that doesn't lessen the sting for Insomniac any, though. Luckily, the Not 50 award categories are fluid, so we have no such problem! Enjoy your Damn Fine Game award, Insomniac – you guys earned it!
Also, I realize my choice of screenshot undermines everything I just said, but I just can't pass up a ridiculously immature fart joke. The game is great though, really.
Girl Power Award: Kassandra (Assassin's Creed Odyssey)
Look, I know girl power is a total cliché; having a tough-as-nails female protagonist who is basically just a gender swap of a male hero isn't innovative or progressive, and it's not particularly good storytelling. If we really want more diverse representation in games then female characters should be more than just ass-kicking heroines – they should be realistic, multidimensional, relatable, and even flawed.
All that said, I'm totally giving Kassandra a great big Girl Power pass this year, because as it turns out, Spartan-kicking soldiers off cliffs and humping your way across Greece is SUPER FUN. If you played through Odyssey with Alexios as your protagonist, you win this year's You Chose Poorly award.
F--- Yarny? F--- YOU Award: Unravel Two
If you're unaware of the backlash over the original Unravel, let me get you up to speed. Unravel was made by a small indie studio called ColdWood Interactive, but since EA published the game – and EA is obviously an evil megacorporation that's run by Satan – the adorable protagonist and heartfelt message about the connections we make in life were clearly just a manipulative ruse to make more money. I mean, it's impossible that anyone at EA could've just seen the game and thought it was a project worth supporting without some dark, ulterior motive that takes advantage of gamers, right? This is EA we're talking about!
Or you could ignore all that noise and just enjoy the damn game. That's what I did, and ended up finding a touching little platformer with some interesting physics-based mechanics and puzzles. Unravel 2, which landed with a thud this year that a yarn doll isn't even capable of making, added co-op into the mix, which elevated the puzzles to a whole new level. I played through Unravel 2 with my wife and it was one of my favorite gaming moments of 2018. If you want to deprive yourself of that experience over your misguided hate of a corporation, I feel sorry for you. Everybody else should try it out.
Best One Night Stand Award: Call of Duty: Blackout Ops 4
I'm not a huge battle-royale player, but I tried out Black Ops 4's new Blackout mode one night shortly after release, once again for the enlightenment of this column's readers (YOU'RE WELCOME). I had a blast, and actually did surprisingly well, landing a second-place finish in my last match. I also haven't gone back to it since. I still have Black Ops 4 installed on my PS4 and want to put more time into the game, but ultimately I'd be okay if I never get back to it. Sometimes one night of intense fun in a video game is enough. I don't really have a joke here, it's just plain true.
Best Open World Award: Red Dead Redemption II – BY FAAAR
This totally seems like it should be a real category in the Top 50, but somehow it's not! Red Dead faced plenty of competition this year, from Assassin's Creed Odyssey's sprawling take on Greece, to Spider-Man's massive rendition of Manhattan. And yet, as much as I love those games, they're not even close to the level of depth, detail, and interactivity of Red Dead's remarkable vistas. So stop trying to fast-travel through them already! Sheesh!
Other than Red Dead's lack of giant naked statues, its open world is just perfect.
Irrational Fear For A Sidekick Award: My Beautiful Horse (Also Red Dead II)
Sure, Palicos are adorable cat companions, but I feel absolutely nothing when a giant monster stomps mine into a pancake. Atreus? He can get his prepubescent butt kicked off a cliff for all I care – I'm not his horribly unqualified father. When it comes to my beloved horse in Red Dead, however, I will park hitch her up a mile away from any mission objective that seems like it could be even remotely dangerous. In fact, I have literally dismounted and put Arthur in the line of gunfire to protect her before!
Part of my irrational concern for my four-legged soulmate comes from not really understanding how Red Dead's horse mechanics work – if my horse dies, will she respawn back in the stables? Or will she gallop her way to horsey heaven? I don't know how it works, and I never will, because my horse is never going to die! Arthur isn't the only one bonding every time I feed her, pet her, and braid her stupid hair!
Best Role-Playing Game Award: MOAR Red Dead!
This actually is a real Top 50 award category – I simply disagree with the group of contestants. We officially named Assassin's Creed Odyssey the best RPG of 2018, which I absolutely supported and still do. However, if we want to talk about actually inhabiting a character and playing a role, Red Dead once again completely blows every other game out of the water.
I went into Red Dead wanting to be the bad guy, a black-hat outlaw who shoots up train coaches and robs banks and guns down the sheriff in front of the whole town while I twirl my moustache because that's what bad guys do for some reason. Instead, I've spent countless hours lugging hale bales around camp and saying howdy to every passing stranger like a yokel, because that's what FEELS right for the character I've transformed into.
Occasionally, my immersive cowboy fantasy is complicated by the fact that missions sometimes force you to do bad things – another complaint I've seemed lodged at the game. However, that criticism fails to recognize that Arthur is also uncomfortable with his actions, but doesn't really have a choice either. In other words, I'M FEELING THE SAME EMOTIONS IN REAL LIFE AS MY CHARACTER IS IN THE GAME AS WE BOTH STRUGGLE WITH THE ACTIONS WE'RE FORCED TO DO. That's insane, and way more affecting than doling out stat points or leveling up gear in what we traditionally think of as an RPG. Can we just get rid of genres entirely at this point?
Dumbest Call Award: NOT Red Dead II?!! (G.I. Staff)
If you saw my list in the Editor Top 10s this year – or just noticed the trend of these last few made-up awards – you probably guessed that I didn't agree with our Game of the Year pick. And in some ways I get it. God of War is a masterpiece. I love the direction Santa Monica Studio took with Kratos, and everything from the writing and voicework to the gameplay and level design are all topnotch. Really, you could shout praise about God of War at my face all day long and I wouldn't complain or disagree with you.
But the sheer ambition and scale of Red Dead is simply on a whole different level, and I've genuinely never played another game like it. Sure, the game had some horrific, spontaneously combusting-horse glitches when it first launched, and there are the *throws up in mouth* pacing issues people have complained about. But it still feels like Rockstar set a new benchmark for video games in 2018, and I don't see it being topped anytime soon.
Also, I got to be a cowboy, dammit!
GOTY Award: Pure Farming 2018
And finally, the category we've all been waiting for! This year's illustrious Goat Of The Year Award is quite the trip…a trip of goats that is! See, that's what a group of goats is called – you know what, never mind, just look at all those adorable baby goats! Pure Farming 2018 wasn't really on my radar before, but now that they've added free goat DLC, it is clearly a game with goats! Congratulations, little cuties, you did it!
Funny To A Point is Jeff Marchiafava's column that celebrates the random, humorous, and downright stupid things that make gaming so much fun. It runs bi-weekly on Fridays.


We are stewards of our customers data and need to do better. <OPINION> I would give us a big fat "F" for data security in 2015.</OPINION> What happened and what needs to be improved? We saw weak passwords, lack of encryption, malware and social engineering over and over again. One very sad aspect of these attacks is once the system was compromised, the attack went on for months, even years prior to the attack being uncovered. So again we really need to do better, reading the logs, doing analytics on system behavior and locking down the data.
High level the attack vectors have not changed much over the years. Malware payloads are still being delivered by drive by downloads and infected emails. Businesses and medical groups are still leaving sensitive data unencrypted, trusted insiders can still get to sensitive information. We are also seeing encrypted connections being made to unknown servers and allowing that traffic to go through our firewalls.
I'm going to do my best to keep my opinion clear by using the <OPINION> </OPINION> tags so you know what my personal opinion is. I'm also not going to go through every attack that happened in 2015. In here I will also let you know what I think should / could have been done to mitigate the attacks.
In IRS's effort to make things easy for users to access their data they exposed very sensitive tax and financial data to hackers. Over 100,000 people were compromised with this system and $50,000,000 is false tax refunds have been stolen from the US Government.
When we design systems, one of the top requirements we have are user experience. If we make it to hard to access the systems they will not be used, make it to easy and the data can be compromised. We need to weigh the value of the data with user experience. The users expect their information to be respected and protected.
UCLA Medical lost 4.5 million records of unencrypted patient data including PII and medical information. There is no excuse to not encrypt sensitive data. I still hear the old excuse of there is a performance impact of encryption. With the availability of hardware encryption modules, this argument does not hold water.
After encrypting data, we still have to be careful about ghost data and data leakage. A DBA can still run database pump and get an unencrypted copy of the data then copy that data to another location. We do this all the time to refresh an environment. Controls need to be placed on data pump copies so any information that is exported from the database will stay encrypted and the location of those copies are known. When moving data from unencrypted to encrypted, all ghost data must be shredded.
Just recently we learned about the Hyatt payment processing data breach. Not much is known at this time other then malware sent encrypted data to an unknown server. This is yet another case of needing to have a trusted path for sensitive data by using white list and denying access to any unknown IP address.
T- Mobile placed their trust in Experian and suffered a massive breach of 15 million customers full name, social security number and date of birth and some passport numbers. In this case no payment card data was compromised. Yet this is still enough information for identity theft. Not a lot of information has been provided on the attack vector used.
This is not an exhaustive list of breaches for 2015.
VTECH the toy manufacturer exposed data on 4.8 million customers due to password insecurity.
Securus lost 70 million call logs and recorded conversations of people in prison. These recordings also included attorney client privileged conversations.
The FBI LEO Portal was hacked, the attack vector and damage is still classified.
Scott Trade lost data on 4.6 million customers under a two year campaign. Krebs on Security reported that the data was used for stock scams.
12) Excellus Blue Cross Blue Shield.
Excellus Blue Cross Blue Shield lost data on 10 million customers. The attack started in 2013 and was not discovered until 2015.
Anthem lost data on 78.8 million customers. I have read the count was actually 80 million customers and 19 million rejected customers.
Secure the data. Encrypt data at rest so if the data is compromised then it will be useless to the criminal.
Encrypt the data on the network when there is sensitive data going through it. Man in the middle attacks happen.
Build trusted paths for sensitive information. All sensitive information must go through that path. If an encrypted session is being built to an unknown server, deny that connection.
Secure the parameter. We are letting encrypted traffic go to unknown servers. This has to stop by using white list. If a workstation or node can process sensitive data, then that workstation or node should not be able to access unknown servers.
Secure programming practices. I still see first hand sloppy programming that is vulnerable to sql injection. Organizations must impalement secure coding practices with code reviews that also include looking to vulnerability. A couple months ago, I came across a piece of code that was vulnerable to sql injection, when I brought it up to program management I was told, going back to fix the problem would put the program behind schedule, move forward and we will fix it after going production. <OPINION> This is the wrong attitude. </OPINION>If the program had standards in place before coding started, then the problem would not have gotten as far as it did.
Secure the data from trusted insiders. I wont get into the political issues of Bradly Manning or Edward Snowden. Both of them were vetted and had access to sensitive information, they broke their trust and stole information that did incalculable damage.
Routinely review audit logs to look for unusual behavior. I'm still seeing audit logs get ignored until there is a problem. Products like Oracle Audit Vault, brings all of your audit into one package where you can create BI dashboards to find out when something is happening that is outside of the norm.


text:
Dr. Ananyo Bandyopadhyay, (MS, ME 2011; PhD, MSE 2012)
During my four years of undergrad in Metallurgical Engineering at Jadavpur University in my hometown of Kolkata, India, I did an internship in a steel foundry where I worked on designing molds of locomotive wheels, side frames and bolsters of freight cars and worked in the metallurgy lab and steel melt shop. Then after my undergrad, I worked as a technical engineer in Vesuvius, India for two years before coming to Michigan Tech to pursue my MS and PhD. At Vesuvius, I worked on manufacturing of refractory used in continuous casting of steel. After my PhD, I joined Advanced Forming Technology (AFT) in Longmont, Colorado where I worked on manufacturing of precision metal parts through powder metallurgy and injection molding. I was doing research on manufacturing, sintering processes of various superalloys, stainless steels and other exotic materials. After working at AFT for 3 years, I was looking forward to doing more research with less involvement in the manufacturing industry. Thus, I pursued a career in railroad research at TTCI which brought me back to the rail industry after 12 years and I have been here for almost 3 years!
At TTCI, we have 52 square miles of area with over 50 miles of different test tracks with varying curvature and grades where cutting edge technological research takes place. The Facility for Accelerated Service Testing (FAST) at TTCI has a 2.7-mile loop where we have our own heavy axle load freight train that runs every night for more than half a year. The AAR sponsored projects include testing of latest generation rails, rail lubrication technologies at FAST and in revenue service of the different Class I freight railroads across USA and Canada.
I manage some of the rail projects at TTCI, including AAR sponsored research as well as commercial projects. My job responsibilities include data collection, failure and fatigue analysis, rail life modeling, metallography along with writing technical articles for journals and magazines. Other engineers assist me in collecting data and conducting experiments. Often, I visit freight railroads and transit agencies when they have rail failure issues and they contact TTCI for solutions.
I cherish my memories at Michigan Tech and feel proud to be an alumnus. The education and tools I learned during my MS and PhD have paved the way for achieving continued success in my research. During my stay at Michigan Tech, I published six journal articles and numerous conference papers and that's something I still continue to do at TTCI.
TTCI


Ultimate Confections - Click here to leave or view comments for this vendor!
Ultimate Confections offers only a great quality chocolate in many different varieties that will serve great for any occasion. You can taste the pride we take in creating our gourmet chocolates. All of our confections are made with the finest ingredients, individually hand-dipped and distinctively packaged. Personal attention is how we create the ultimate confection.


Yesterday I had the pleasure of taking family portraits for our very dear friends Sean, Darcy, Aiden, and Toby! Sean, Darcy, Stan, and I have been friends for a very long time. Their son Aiden was born almost a year ago and Stan and I have been in love with him ever since! And let's not forget Toby, who is Buddy's best friend in the whole universe. We had so much fun yesterday and I know that Darcy is DYING for me to get these pictures up, so here they are!
Jessica, June 9th - please send me your email address!
Just a little Public Service Announcement: Jessica, I saw that you contacted me through my website, but it looks like you accidentally left off the "@_____________" part of your email address, so all I have is your name and wedding date. If you could email me at stacy@stacycross.com, that would be fabulous!
Hope everyone had a great Thursday.....busy, busy day at the hospital today (I have my nursing school clinicals on Thursdays and Fridays) but I'm loving it! :) I've got some exciting photo ops coming up this weekend, so stay tuned for some pics.
Oh, PS - If you live in Austin and haven't been to Austin Java, this is my official "It is so so so so great" plug for this place! I'm sitting here as I write and I'm enjoying their free Wi-Fi and bottomless cup of coffee for $ 2.29 (now you know why this post is so long - I'm totally wired!)and later I will be having some awesome lunch off of their very extensive menu. Everything I've ever gotten to eat here has been fabulous and they have tons of healthy options! Anyway, you should defintely check it out!
Don't tell my mom and dad.....they'll blush!
Unfortunately, a lot of the really great work I've been doing lately is stuff that I can't (and won't) post on my blog or my website, or anywhere else for that matter. That's because I've been doing a lot of "Boudoir" shots lately and they've turned out amazingly! Lots of brides have asked about these shoots so that they can have these pictures made for their husbands as a wedding present - very fun! Well, I also got the opportunity to do some of these shoots for some ladies that had been married for a long time, but wanted to get their husbands some great pictures for Valentine's Day - also fun and cute!
"People will not always believe what you say, but they will always believe what you do."
I love this because it challenges me to walk my talk....and live the life I aspire to live - RIGHT NOW.
The other day I got to shoot some unofficial 2 week pictures of sweet little Ben. You may remember the newborn pictures I posted of him - well, I will be doing a "First Year" package for Ben and his parents, so we'll all get to see Ben grow during his first year together!
I know that there have to be a lot of great husbands out there....but mine really is the best. Stan is the most kind, loving, intelligent, handsome, fun, silly, and giving person I've ever known - I completely got the jackpot when I married him! There is literally nobody on earth that I'd rather be around.....and nobody who could ever love me like he does. He's incredible.
Part of the fun thing about being a married lady AND a wedding photographer is that I understand the depth of what my brides and grooms are embarking on - and it's still special everytime I get to witness that in someone's life. When Stan and I shoot together, there's always a moment during the ceremony when we look at each other and smile.....I think it's our little way of remembering our own committment - and how thankful we are that we made it!
Stan, I love you with all my heart - Happy Valentine's Day!
Where'd they all come from?!?!


text:
NYU museum studies
For three decades, the Program in Museum Studies at New York University has offered an innovative course of study in the contemporary theory and practice of museum work. Emphasizing both the interdisciplinary study of museums and courses of practical training, the program has prepared more than three hundred graduates for positions of increasing responsibility in museums throughout the world. Our graduates are working in museums of fine art, history, anthropology, technology, and natural history; in arboretums, national parks, and science centers; with private and corporate collections; and in government agencies, historical societies, and art galleries. And their careers cover the full range of museum activity, working as directors, curators, educators, registrars, collection managers, and development, media and public relations specialists.
In addition to obtaining a broad foundation in the history and theory of museums, the creation and maintenance of exhibitions and collections, and museum management, our students pursue personal programs of study supporting individual academic and career objectives. Museum Studies at NYU actively crosses disciplinary boundaries in two ways, assembling scholars from many academic fields whose work engages museum theory and practice, and training students to perform a variety of roles within all types of museums. Located in New York City, the cultural capital of the world, and within a great university known for its interdisciplinary orientation, NYU Museum Studies offers an unparalleled opportunity to prepare for the challenges of museum work in this new century.
The Program in Museum Studies at New York University offers an innovative course of study in the contemporary theory and practice of museum work. Emphasizing both the interdisciplinary study of museums and courses of practical training, the program has prepared more than three hundred graduates for positions of increasing responsibility in museums throughout the world.
In addition to pursuing further degrees, alumni of the Program currently hold positions at the following institutions:
ICI (Independent Curators International)
1stdibs.com
Alaska Community Foundation
Arab Youth Venture Foundation
Art Preservation Services
Boston Public Schools
Brooklyn Historical Society (2)
California Historical Society
California State Parks Foundation
Camp Alonim
Dallas Heritage Village
Dedalus Foundation
Eva Dean Dance
Friends of the High Line
Granite Club
Historic Albany Foundation
Houston Museum of Natural Science
Intrepid Sea, Air and Space Museum
Lehmann Maupin Gallery
Lower East Side Tenement Museum
Masterpiece International Ltd.
Matheson & Howell PC
Merchants House Museum (2)
Mississippi Department of Archives and History
Morgan Library & Museum
Morris-Jumel Mansion
Museum of American Finance (2)
Museum of the City of New York
New-York Historical Society
Institute for the Study of the Ancient World at New York University
Outset Contemporary Art Fund
Penn Museum
Pentagram Design (2)
Philips Auctioneers
Point Foundation
Princeton University Art Museum
Rocky Mountain National Park/National Park Service
Ropes & Gray LLC
Rubin Museum of Art
San diego history center
San Diego Regional Airport Authority
Shea Art Advisory Inc.
Sheldon Museum of Art
SOAS, The University of London
Solomon R. Guggenheim Museum (2)
Spain Arts and Culture
Tel Aviv Museum of Art, American Friends
The Asian Art Museum
The Gilder Lehrman Institute of American History
The Hall at Patriot Place presented by Raytheon
The Jewish Musuem
The Metropolitan Museum of Art (4)
The Morgan Library and Museum
The Robert Mapplethorpe Foundation, Inc.
Wexner Center for the Arts
Women's Refugee Commission
Yale University Art Gallery
Want to learn more about an alum/alumna at one of these institutions? Please contact Aimee VonBokel, Ph.D. for further information at aimee.vonbokel@nyu.edu.
Houda Lazrack '16 interned at the Rubin Museum in 2015 and ran into a familiar face!
Several students interned at the Whitney Museum during the construction of their new building.


They called their kids what?!
Congratulations to Rose Byrne and Bobby Cannavale, who have welcomed their second child together.
With People confirming the happy news, no other information is known about the new addition to the family – Rose and Bobby are notoriously private about their personal life.
The 38-year-old Aussie actress and the American star already share 21-month-old son Rocco Robin together, and Bobby also has a 22-year-old son named Jake from a previous marriage to actress and screenwriter Jenny Lumet, 50.
Rose first announced her second pregnancy in Jones earlier this year.


Here posted profiles refined whores Manly West, thirsting to surrender to you for specified price.
Sweetheart babe, I want carefree dating with the continuation. Right now, dial my phone number and come, I'll do anything, as you wish!
Whores Manly West these are professional women, who know everything ways satisfaction men.


text:
Biografia
Nato a Taipei col nome di Chien Fu-Nan, immigrò in Canada e lì gli venne dato il nome Fernando Chien. Ha praticato Taekwondo, Karate e Hung Gar.

Filmografia

Attore

Cinema
 La spia, regia di Jim Donovan (1998)
 Falso Indizio, regia di Douglas Jackson (1998)
 L'arte della guerra, regia di Christian Duguay (2000)
 Soldier Of God, regia di W.D. Hogan (2005)
 Honor, regia di David Worth (2006)
 TKO, regia di Declan Mulvey (2007)
 Beyond The Ring, regia di Gerson Sanginitto (2008)
 La mummia - La tomba dell'Imperatore Dragone, regia di Rob Cohen (2008)
 Angel Of Death, regia di Paul Etheredge (2009)
 Bunraku, regia di Guy Moshe (2010)
 Fast & Furious 5, regia di Justin Lin (2011)
 Warrior, regia di Gavin O'Connor (2011)
 Red Dawn - Alba rossa, regia di Dan Bradley (2012)
 Iron Man 3, regia di Shane Black (2013)
 Transcendence, regia di Wally Pfister (2014)
 The Accountant, regia di Gavin O'Connor (2016)
 The Concessionaries Must Die!, regia di America Young (2017)
 Sun Dogs, regia di Jennifer Morrison (2017)
Unspoken: Diary of an Assassin, regia di Ron Yuan (2017)
Rogue Warfare, regia di Mike Gunther (2019)
Rogue Warfare: The Hunt, regia di Mike Gunther (2019)
Rogue Warfare 3: Death of a Nation, regia di Mike Gunther (2020)
Shang-Chi e la leggenda dei dieci anelli, regia di Destin Daniel Cretton (2021)

Televisione
 Student Bodies - serie TV, 2 episodi (1998)
 Gli Specialisti - serie TV, episodio 2x15 (1999)
 The Secret Adventures of Jules Verne - serie TV, episodio 1x11 (2000)
 Il Segno dei Quattro - telefilm (2001)
 Star Trek: Enterprise - serie TV, 2 episodi (2004)
 Crossing Jordan - serie TV, episodi 4x13 (2005)
 My Own Worst Enemy - serie TV, episodio 1x02 (2008)
 The Guild - webserie, 9 episodi (2009)
 Melrose Place - serie TV, episodio 1x04 (2009)
 NCIS: Los Angeles - serie TV, episodio 1x16 (2010)
 The Legend Of Neil - serie TV, episodio 3x07 (2010)
 The Cape - serie TV, episodio 1x01 (2011)
 True Justice - serie TV, episodio 1x10 (2011)
 The Indestructible Jimmy Brown - telefilm (2011)
 The Division - serie TV, episodio 1x03 (2012)
 Ray Donovan - serie TV, episodio 1x10 (2013)
 We Are Fathers - serie TV, 2 episodi (2016)
 The Last Ship - serie TV, 6 episodi (2016)
 NCIS: New Orleans - serie TV, episodio 3x19 (2017)
Iron First - serie TV, 7 episodi (2018)
FBI - serie TV, episodio 2x03 (2019)
Hawaii Five-0 - serie TV, 5 episodi (2019-2020)
MacGyver - serie TV, episodio 4x02 (2020)
The Blacklist - serie TV, 2 episodi (2021)

Doppiatore

Cinema

 Il piccolo Yeti (Abominable), regia di Jill Culton (2019)

Televisione

 We Bare Bears - Siamo solo orsi (We Bare Bears)- serie TV animata, episodio 3x02 (2017)
 American Dad! - serie TV, episodio 13x16 (2019)

Videogiochi
Spider-Man (2018)

Doppiatori italiani
Nelle versioni in italiano delle opere in cui ha recitato, Fernando Funan Chien è stato doppiato da:
 Raffaele Palmieri in Warrior
 Riccardo Scarafoni in The Last Ship
 David Chevalier in The Accountant

Note

Collegamenti esterni

redpajama_set_name: RedPajamaWikipedia


text:
Q: pyspark dataframes: Why can I select some nested fields but not others? I'm trying to write some code to un-nest JSON into Dataframes using pyspark (3.0.1) in Python 3.9.1.
I have some dummy data with a schema as follows:
data.printSchema()
root
 |-- recordID: string (nullable = true)
 |-- customerDetails: struct (nullable = true)
 |    |-- name: string (nullable = true)
 |    |-- dob: string (nullable = true)
 |-- familyMembers: array (nullable = true)
 |    |-- element: struct (containsNull = true)
 |    |    |-- relationship: string (nullable = true)
 |    |    |-- name: string (nullable = true)
 |    |    |-- contactNumbers: struct (nullable = true)
 |    |    |    |-- work: string (nullable = true)
 |    |    |    |-- home: string (nullable = true)
 |    |    |-- addressDetails: array (nullable = true)
 |    |    |    |-- element: struct (containsNull = true)
 |    |    |    |    |-- addressType: string (nullable = true)
 |    |    |    |    |-- address: string (nullable = true)

When I select fields from familyMembers I get the following results as expected:
data.select('familyMembers.contactNumbers.work').show(truncate=False)
+------------------------------------------------+
|work                                            |
+------------------------------------------------+
|[(07) 4612 3880, (03) 5855 2377, (07) 4979 1871]|
|[(07) 4612 3880, (03) 5855 2377]                |
+------------------------------------------------+

data.select('familyMembers.name').show(truncate=False)
+------------------------------------+
|name                                |
+------------------------------------+
|[Jane Smith, Bob Smith, Simon Smith]|
|[Jackie Sacamano, Simon Sacamano]   |
+------------------------------------+

Yet when I try to select fields from the addressDetails ArrayType (beneath familyMembers) I get an error:
>>> data.select('familyMembers.addressDetails.address').show(truncate=False)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.9/site-packages/pyspark/sql/dataframe.py", line 1421, in select
    jdf = self._jdf.select(self._jcols(*cols))
  File "/usr/local/lib/python3.9/site-packages/py4j/java_gateway.py", line 1304, in __call__
    return_value = get_return_value(
  File "/usr/local/lib/python3.9/site-packages/pyspark/sql/utils.py", line 134, in deco
    raise_from(converted)
  File "<string>", line 3, in raise_from
pyspark.sql.utils.AnalysisException: cannot resolve '`familyMembers`.`addressDetails`['address']' due to data type mismatch: argument 2 requires integral type, however, ''address'' is of string type.;;
'Project [familyMembers#71.addressDetails[address] AS address#277]
+- LogicalRDD [recordID#69, customerDetails#70, familyMembers#71], false

I'm confused. Both familyMembers and addressDetails are ArrayTypes, yet selecting from one works but not the other. Is there an explanation for this, or something I've missed? Is it because one is nested within the other?
Code to reproduce (with just 1 record):
from pyspark.sql.types import StructType
from pyspark.sql import SparkSession, DataFrame
import json

rawdata = [{"recordID":"abc-123","customerDetails":{"name":"John Smith","dob":"1980-04-23"},"familyMembers":[{"relationship":"mother","name":"Jane Smith","contactNumbers":{"work":"(07) 4612 3880","home":"(08) 8271 1577"},"addressDetails":[{"addressType":"residential","address":"29 Commonwealth St, Clifton, QLD 4361 "},{"addressType":"work","address":"20 A Yeo Ave, Highgate, SA 5063 "}]},{"relationship":"father","name":"Bob Smith","contactNumbers":{"work":"(03) 5855 2377","home":"(03) 9773 2483"},"addressDetails":[{"addressType":"residential","address":"1735 Fenaughty Rd, Kyabram South, VIC 3620"},{"addressType":"work","address":"12 Haldane St, Bonbeach, VIC 3196 "}]},{"relationship":"brother","name":"Simon Smith","contactNumbers":{"work":"(07) 4979 1871","home":"(08) 9862 6017"},"addressDetails":[{"addressType":"residential","address":"6 Darren St, Sun Valley, QLD 4680"},{"addressType":"work","address":"Arthur River, WA 6315"}]}]},]
strschema = '{"fields":[{"metadata":{},"name":"recordID","nullable":true,"type":"string"},{"metadata":{},"name":"customerDetails","nullable":true,"type":{"fields":[{"metadata":{},"name":"name","nullable":true,"type":"string"},{"metadata":{},"name":"dob","nullable":true,"type":"string"}],"type":"struct"}},{"metadata":{},"name":"familyMembers","nullable":true,"type":{"containsNull":true,"elementType":{"fields":[{"metadata":{},"name":"relationship","nullable":true,"type":"string"},{"metadata":{},"name":"name","nullable":true,"type":"string"},{"metadata":{},"name":"contactNumbers","nullable":true,"type":{"fields":[{"metadata":{},"name":"work","nullable":true,"type":"string"},{"metadata":{},"name":"home","nullable":true,"type":"string"}],"type":"struct"}},{"metadata":{},"name":"addressDetails","nullable":true,"type":{"containsNull":true,"elementType":{"fields":[{"metadata":{},"name":"addressType","nullable":true,"type":"string"},{"metadata":{},"name":"address","nullable":true,"type":"string"}],"type":"struct"},"type":"array"}}],"type":"struct"},"type":"array"}}],"type":"struct"}'

spark = SparkSession.builder.appName("json-un-nester").enableHiveSupport().getOrCreate()
sc = spark.sparkContext

schema = StructType.fromJson(json.loads(strschema))

datardd = sc.parallelize(rawdata)
data = spark.createDataFrame(datardd, schema=schema)

data.show()
data.select('familyMembers.name').show(truncate=False)
data.select('familyMembers.addressDetails.address').show(truncate=False)

A: To understand this you can print the schema of :
data.select('familyMembers.addressDetails').printSchema()

#root
# |-- familyMembers.addressDetails: array (nullable = true)
# |    |-- element: array (containsNull = true)
# |    |    |-- element: struct (containsNull = true)
# |    |    |    |-- addressType: string (nullable = true)
# |    |    |    |-- address: string (nullable = true)

See here you have an array of arrays of structs which is different from the initial schema you have. So you can't directly access address from the root, but you can select the first element of the nested array then access the struct field address :
data.selectExpr("familyMembers.addressDetails[0].address").show(truncate=False)

#+--------------------------------------------------------------------------+
#|familyMembers.addressDetails AS addressDetails#29[0].address              |
#+--------------------------------------------------------------------------+
#|[29 Commonwealth St, Clifton, QLD 4361 , 20 A Yeo Ave, Highgate, SA 5063 ]|
#+--------------------------------------------------------------------------+

Or:
data.select(F.col('familyMembers.addressDetails').getItem(0).getItem("address"))

A: Along with the answer that @blackbishop provided You can also use the combination of the select and expr to get the output as below :
data.select(expr('familyMembers.addressDetails[0].address')))

Output :

You can also use explode to get all the addresses if you want as below :
data.select(explode('familyMembers.addressDetails')).select("col.address")

Output :

redpajama_set_name: RedPajamaStackExchange