Reflection - Learning Launch

Olivier Saidi

Challenge

I was leading a project to build and implement “Technology Competency Framework” for my  organization’s large software programmers’ pool of 0 to 4 years of experience in the company. We have  identified 6 different Lines of Technologies (LOTs) which are prevailing for the business to categorize the  target population within. I wanted each LOT to have definite curriculum and associated evaluation at 3  different levels (L1, L2, L3) with increasing expertise and complexity to enable employees gain higher skills  and competencies as they progress year-on-year. We also wanted to map this framework to employees’  career progression within the organization, so it was far more important to have an accurate technology  competency framework in place. Especially “Evaluation” mechanism at every L1, L2, L3 levels for all LOTs.

Employee evaluation framework was decided to be thru’ online assessments in every LOT, based on the  curriculum set for every level (L1, L2, L3). Online assessment was to be using variety of Multiple Choice  Questions (MCQs) of low, medium, high complexities in each level’s assessment. Sizable question bank  was generated and uploaded in the assessment engine. Parameters like Questions’ complexity,  randomization, points for correct answers, passing percentage have been configured in the assessment  engine.

Selection

Overall Evaluation model was conceptualized as stated above and validated to proceed with. Now the  most crucial part of the evaluation framework was ahead of us to establish its authentic workability. For  which “Calibrating the Assessments” was essential, i.e. to ensure that these assessments at each level  would produce the results which are in line with employees’ actual skill levels. For example, when  employee is certified at Level 2 post the assessment, in reality, he/she is indeed at the skill/competency  Level 2 and not at 1 or 3. For this, I’ve decided to apply “Learning Launch” of design thinking.

Application

Selected a group of participants from the target audience and after imparting the necessary training to  them from their respective LOTs asked them to take the online assessments. Then their assessment results  were compared against their professional academic scores when they joined the organization, the scores  they have got in various relevant trainings they underwent after joining the organization and also against  the ratings they used to get on their technical skills and expertise by their supervisors during half yearly  and yearly appraisal process. Control graphs were plotted and cases falling outside of control limits were  analyzed. Initially we found higher numbers out of control limits. Then we adjusted the parameters in the  assessment engine and also modified the question bank repository.

This cycle was repeated 2-3 times with different set of employees to finetune the assessment mechanism  and calibrate the assessments till the cases falling out of control limits were minimal and our solution  became authentic and workable for the organization wide implementation.

Insight

“Learning Launch” tool turned out to be highly effective in this piece of our project. It was crucial and  most challenging work piece of the overall project and if we would have directly implemented the  framework across the organization, without having tested its accurate workability on smaller groups,  2-3  times, the entire initiative would have been a flop show. Learning Launch tool helped us to

-  ensure solution workability in a shorter time,   -  making us take fast corrective actions to calibrate the assessments,   -  minimal but direct involvement of the end users of varied samples covering the larger spectrum of  technologies and levels   -  and most of all, with very minimal efforts, it enhanced our (project team), sponsor’s and key  stakeholders’ confidence as to whatever solution / framework we are providing is the top notched  realistic solution for the benefit of the entire organization and target population.

Approach

I think, every tool has its own pros & cons. Not every tool will help or provide same benefits in different  challenges. According to me, “Learning Launch” is the better one to ensure the workability of the solution,

-  when there are multiple unknown factors involved in the new initiative,   -  when there is no previous data available to cross verify the practicality of the solution,  -  the volume of impacted population is high or very high  -  implementation cost is high or very high

If your challenge doesn’t involve any of the aforesaid factors, then the risk of direct big-bang  implementation is comparatively less and that risk you can mitigate by analyzing the past data and trends  from the similar challenges and decide the course of action on your challenge.


Reflection - Learning Launch Olivier Saidi Challenge I was leading a project to build and implement “ Technology Competency Framework ” for my organization ’s large software programmers ’ pool of 0 to 4 years of experience in the company . We have identified 6 different Lines of Technologies ( LOTs ) which are prevailing for the business to categorize the target population within . I wanted each LOT to have definite curriculum and associated evaluation at 3 different levels ( L1 , L2 , L3 ) with increasing expertise and complexity to enable employees gain higher skills and competencies as they progress year - on - year . We also wanted to map this framework to employees ’ career progression within the organization , so it was far more important to have an accurate technology competency framework in place . Especially “ Evaluation ” mechanism at every L1 , L2 , L3 levels for all LOTs . Employee evaluation framework was decided to be thru ’ online assessments in every LOT , based on the curriculum set for every level ( L1 , L2 , L3 ) . Online assessment was to be using variety of Multiple Choice Questions ( MCQs ) of low , medium , high complexities in each level ’s assessment . Sizable question bank was generated and uploaded in the assessment engine . Parameters like Questions ’ complexity , randomization , points for correct answers , passing percentage have been configured in the assessment engine . Selection Overall Evaluation model was conceptualized as stated above and validated to proceed with . Now the most crucial part of the evaluation framework was ahead of us to establish its authentic workability . For which “ Calibrating the Assessments ” was essential , i.e. to ensure that these assessments at each level would produce the results which are in line with employees ’ actual skill levels . For example , when employee is certified at Level 2 post the assessment , in reality , he / she is indeed at the skill / competency Level 2 and not at 1 or 3 . For this , I ’ve decided to apply “ Learning Launch ” of design thinking . Application Selected a group of participants from the target audience and after imparting the necessary training to them from their respective LOTs asked them to take the online assessments . Then their assessment results were compared against their professional academic scores when they joined the organization , the scores they have got in various relevant trainings they underwent after joining the organization and also against the ratings they used to get on their technical skills and expertise by their supervisors during half yearly and yearly appraisal process . Control graphs were plotted and cases falling outside of control limits were analyzed . Initially we found higher numbers out of control limits . Then we adjusted the parameters in the assessment engine and also modified the question bank repository . This cycle was repeated 2 - 3 times with different set of employees to finetune the assessment mechanism and calibrate the assessments till the cases falling out of control limits were minimal and our solution became authentic and workable for the organization wide implementation . Insight “ Learning Launch ” tool turned out to be highly effective in this piece of our project . It was crucial and most challenging work piece of the overall project and if we would have directly implemented the framework across the organization , without having tested its accurate workability on smaller groups , 2 - 3 times , the entire initiative would have been a flop show . Learning Launch tool helped us to - ensure solution workability in a shorter time , - making us take fast corrective actions to calibrate the assessments , - minimal but direct involvement of the end users of varied samples covering the larger spectrum of technologies and levels - and most of all , with very minimal efforts , it enhanced our ( project team ) , sponsor ’s and key stakeholders ’ confidence as to whatever solution / framework we are providing is the top notched realistic solution for the benefit of the entire organization and target population . Approach I think , every tool has its own pros & cons . Not every tool will help or provide same benefits in different challenges . According to me , “ Learning Launch ” is the better one to ensure the workability of the solution , - when there are multiple unknown factors involved in the new initiative , - when there is no previous data available to cross verify the practicality of the solution , - the volume of impacted population is high or very high - implementation cost is high or very high If your challenge does n’t involve any of the aforesaid factors , then the risk of direct big - bang implementation is comparatively less and that risk you can mitigate by analyzing the past data and trends from the similar challenges and decide the course of action on your challenge .