Sell on Souq.com with CedCommerce.
Souq.com is a leading e-commerce Web site in the Middle East and North Africa region (MENA). It offers a host of options for buyers and sellers to transact through an online secure environment to purchase just about everything, either from our online retail stores, or through the Marketplace.
Offering 8.4 million products, Dubai-based Souq.com is the largest online retailer in the Arab world, operating in English and Arabic.
What is Souq Integration from CedCommerce?
It establishes a real-time channel through API Integration to feed product critical data to Souq.com. This integration offers a platform for online sellers of Magento 1 and Magento 2, Woocommerce, Shopify, BigCommerce, 3dCart, Volusion, and so on to offer their product for sale on Souq.com.
What all can be done and managed through Souq Integration By Cedcommerce ?
Get your product listed on the marketplace.
All the orders get managed from your store panel.
If product does not get uploaded, error notification get listed on the store.
Informs about the status of the product, whether LIVE or UNDER REVIEW.
Get bulk product uploaded in one go.
Orders get fulfilled in a single step.
It allows you to upload data in one-go directly at Souq.com. It is accomplished with centralized Status update ,sends dynamic alerts and compliance reports for the entire product fulfilment cycle. CedCommerce helps in Uploading Products, expert training to make you familiar with navigation and operation. Also, it offers expert advice and support for hassle-free operation, better product discover-ability and higher sales.


Cheatscape is the best place to find cheats for Runescape and World of Warcraft. Cheatscape is the primary online source for the latest and best cheats available for all MMORPG games.
Pass1000 : Successful Guide students to pass any IT certification. Provide latest Exam Topics (dumpandruns).
Crime Minds is a friendly forum where people interested in current criminal cases can exchange information and discuss the latest news, research, evidence and ideas.
Free forum : Keep up to date with all the latest Doctor Who Merchandise releases aha join our friendly community!!!!!
monkey, room, monkeying, around, catching, #latest, news!
Free forum : this is a forum for movie loving people. latest movie review, movie download, discuss about a movie.
Free forum : This is the site for EVERYTHING and ANYTHING you want to talk about. Want to discuss the latest games? Done. Want to have fun? Done. This is THE place to have a great time.
forum, bearville, globe, great, good, #latest, news, babv!
Free forum : Sharing & Joining the World Together Get daily Ghana lotto and football betting forcast tips, Free browsing, Fta, issues and all the latest news.
Find the best forums and communities for latest!


WASHINGTON: Unites States (US) Principal Deputy Assistant Secretary Alice Wells has said that the democratic transition of power in Pakistan was a great achievement and the US would like to see the process succeed.
"We want civilian institutions to strengthen, and it's in that vein that when we engage with Pakistani leaders we express concern when there is evidence or indications that civil society or media are not able to participate fully or without pressure, because that will ultimately detract from what should be a success story in Pakistan," she added.
The principal deputy assistant secretary also said that Pakistan had lost "many tens of thousands of lives in beating back the Pakistani Taliban from Federally Administrated Tribal Areas (FATA) areas".


text:
Atholton Website
Entire Site ArtsAtholt-OnionEditorialFeaturesGreen TeamNewsOpinionsPodcastsReviewSportsUncategorizedVideos
FeaturesNewsOpinionsSportsUncategorizedArtsEditorialReviewVideosAtholt-OnionPodcastsGreen Team
AtholtonAtholton High SchoolSchoolHoward CountySportsHCPSSAHSNewsfunartMusicLifeAtholton Raiderstop storymoviestechnologyFeatureshomecomingraidersfootball
January 2023 December 2022 November 2022 October 2022 September 2022 June 2022
Atholt-Onion
Welcome to Atholton High School's top site for breaking news affecting our community!
NextE & I Tea Time Episode 5: Binge-Worthy Christmas Specials
A Look Into the Choir Life at Atholton
Do you love singing? Are you interested in taking choir? Take a deep dive into the life of an choral student.
jhoward01 on December 7, 2018
Jackson Howard
What is that beautiful sound coming from room B118? Well, it's this year's choirs preparing for their winter concert on December 18th at 7:00pm in the auditorium. Now you may think that chorus is an easy A, but let's set it straight. If you're looking at taking choir for the 2019-20 school year, then you might want to know that choir is a place where you must put forth your best effort if you want to be a great choral singer. If you are wondering what it takes to prepare for a concert than you have come to the right place.
Each year, choir, band and orchestra have three annual concerts. They are fall, winter and spring concerts. For the fall concert, all three ensembles perform one or two of their own pieces. Towards the end of the concert the marching band comes in to do a few songs and then sing the Atholton fight song with the audience. The fall concert is the only one where every ensemble performs on the same night. For example, for the winter concert each ensemble performs on their own night. This year, band performs on Wednesday, December 12th, and the orchestra performs the next night on the 13th.
Junior classmen, Elena Wenderoth, explained how choir typically prepares for their concerts. "As we approach the date of the concert, we start running through the program and focus less on sight-reading and learning notes. In the evenings, there are more rehearsals for specialized choirs followed by the combination of all choirs." As Elena explained, preparing for a concert usually ramps up a week or two after the previous concert, so there is not a lot of time to relax, as they start to prepare for another one.
This year's concert has a very special theme that people should pay close attention to: Mental Health Awareness. For a choir it is very important to show emotion, especially when you have a theme that is as serious as mental health. "We do a lot of movement and that is not just for expressions, but also it helps us sing better and release tension and helps with rhythm and study beat," said Ms. Meloro, an advocate of making sure that music is able to be transmitted to the audience, and what it is about. David Lowe, junior classmen, gave his intake on how he believes choir connects with the audience,
"We try to make the music connect to the audience by picking something that they can relate to, or something that can be applied to current situations or things happening in the real world."
Coming up with a theme or songs for a concert is very tricky because Ms. Meloro has to be conscientious of whether a song or theme is too religious or even offensive to some people. Ms. Meloro talked about how "it is difficult in some ways," but she believes that "a good teacher is conscious about a good balance or having a good educational reason for programming it." One of the pieces that choir is doing is "City Called Heaven", which is thinking about "going through a hard time and you know that if you just bare through it, you'll have the reward which is heaven," said Ms. Meloro. She explained how "someone who may not believe in heaven or have that subscription they can still think about getting through a troubled time and thinking of a better place whether its if I just get through midterms, my family is going on a vacation."
Now for those of you who are debating about joining choir, you may be wondering how you would fit within the choir itself. Junior Joelle Popa, in her 3rd year in the Atholton choir, helped provide some insight on how Ms. Meloro makes sure you feel right at home. "Ms. Meloro is very good at listening to people's voices and finding a good place vocally for everyone," she said. Joelle has also been joyful for all the connections and friendships she has made through choir itself alone. "I feel free and welcome to talk to absolutely anyone. My first and longest-lasting high school friendships have been because of choir, and I'm still making new friendships in there everyday," she said. So even if you have more questions, you at least know that choir will be a place where you will feel welcomed and will be able to build relationships with other choirmates.
Choir is a place where you will be counted on to do your part and participate on a daily basis. Ms. Meloro makes sure that students are held accountable, but she does not want students to be discouraged to sing out loud and become scared of joining choir or staying in it. "For learning their parts, we do part testing to make sure you have the accountability to go home and work on it, so they submit a recording of them singing the song that I assign a part test for," as stated by Ms. Meloro, as she describes how to make sure that students are practicing their music. "We also do in class part testing where the whole section will sing and I have the training to pick out certain voices and tell if something is wrong or I can't hear someone."
To finish off this behind-the-scenes look into the choir life, here are some words of encouragement from Ms. Meloro herself about what she would say to students who want to join choir. "Come, try, signup. You have my blessing and encouragement. You'll learn a lot and I think that is the biggest thing," described Ms. Meloro. "It is hard work but it can be so rewarding when you do the final product and you hear yourself sing and reflect on what you've done."
Posted in: Arts
Tagged in: Atholton, Choir, Howard County Public School System, Music, Music Boosters, winter concert
Posted by jhoward01
Jackson is a junior at Atholton High School. He loves to write when he has the opportunity. Also he enjoys watching sports and playing sports. His profession that he wants to become is a sports journalist for the Washington Post or the Associated Press. He spends his free time by playing with his two dogs Eden and Harper and binge watches NCIS, Parks and Recreation and The Office.
The Lasting Impact Of Biggie Smalls
Mikey Williams First Season at Vertical Academy
Spirit Week Day 4: Wacky Tacky Day
TBH Episode 3: New Snow Day Policy
Spirit Week Day 5: Color Day & Pep Rally
The Transgender Bathroom Debate Continues...
Introducing Non-Binary Artists in the Music Industry
E & I Tea Time Episode 5: Binge-Worthy Christmas Specials
How 21st Century Standards Are Affecting Teens' Mental Health
What People Forget about Martin Luther King Jr. January 13, 2023
Joy To The World, A Concert Has Come January 9, 2023
Dream Big With Atholton Dance Company: The Winter Dance Concert January 3, 2023
Shaking it up with POMs January 3, 2023
Humans of Atholton
atholtonnews.com

redpajama_set_name: RedPajamaCommonCrawl


French toast is my breakfast weakness, it genuinely makes me go weak at the knees (or stomach)! Whatever you call it – French toast, pain perdu, gypsy toast or eggy bread – if it's on the menu I'm most likely going to choose it.
I love French toast mainly because there are so many variations. This simple breakfast (or supper, I'm not here to judge) can be accompanied by crisp, salty bacon, sharp berries, or indulgent clotted cream. Not to mention a favourite tea-time staple - tomato ketchup.
This recipe for orange French toast takes your average eggy bread to another level. A perfect weekend brunch which looks more impressive than it actually is. I like to call it my Paddington Bear Breakfast, thanks to the lashings of marmalade I spread on the freshly cooked bread which melts into a sticky syrup. Delicious!
Whisk two eggs in a bowl with a heaped tsp of ground cinnamon and a tbsp of light brown sugar. Cut an orange in half and give it a good squeeze into the egg mixture – but don't use all of the juice or it will become too wet. Then finely add about a tsp of orange zest into the bowl.
Pour the mixture onto a shallow bowl or plate and place two stale pieces of white bread (fresh slices will disintegrate!) into the egg and allow to it absorb the mixture – turning now and again – for about five minutes.
Maybe don't eat anything else for the rest of the day...?
Melt a knob of butter in a pan until it starts to foam, then add the slices of bread to the pan. Fry off in the butter until golden and flip to cook the other side. It takes a couple of minutes each side, so keep an eye on it.
Serve straight away with a good helping of your favourite marmalade.


text:
New Book on Tolkien, Lewis, and Williams
The Lutterworth Press have announced the upcoming release of their book The Ideal of Kingship in the Writings of Charles Williams, C.S. Lewis, and J.R.R Tolkien: Divine [...]
Oloris Publishing seeks art submissions depicting armour, weapons, battle scenes and maps for new Tolkien non-fiction book
Early next year, Oloris Publishing will release Glorious Warriors and That Which They Defend: War and Politics in Middle-earth by Timothy R. Furnish, Ph.D. Applying [...]
Columbia University acquires rare Ballantine Books collection
Columbia University's Rare Book & Manuscript Library has announced Betty Ballantine's gift of a collection of books and papers relating the work of her and [...]
Jef Murray "Middle-Earth" collection of fiction, non-fiction and artwork from Oloris Publishing
The much-anticipated inaugural book from Oloris Publishing, Seer: A Wizard's Journal, is now available. Jef Murray's first full-length title is a collection of [...]
Fantasy Short Stories – new Heroic, Epic and High Fantasy 'zine
Fantasy Short Stories is a new publication established with the aim of publishing the best short stories in Heroic, Epic, and High Fantasy, and with plenty of Swords and [...]
HarperCollins to publish official tie-ins to The Hobbit films
HarperCollins has acquired exclusive worldwide publishing rights from Warner Bros Consumer Products for tie-in books to the two highly anticipated films, The Hobbit: An [...]
Tolkien expert to visit Birmingham SF Group
For its April meeting this year the Birmingham Science Fiction Group is proud to welcome Robert Blackham, noted Tolkien expert, who will be showing his new presentation about [...]
World FCon 2013 announces Artist Guest of Honour, Alan Lee
World Fantasy Convention 2013 is pleased to announce that Academy Award- and World Fantasy Award-winning illustrator Alan Lee will be its Artist Guest of Honour in Brighton. [...]


text:
Let's focus on the real foreign property ownership problem
Following on from the second half of the last decade, the first few weeks of...
Six common 'crimes of modern-day call handling' are blighting UK estate and letting agencies and...
As I look towards this year, I am sure that my thoughts are the same...
2020 - the year of Airbnb, and the opportunity for agents
Over the past decade, the short-term rental industry, now eponymised by Airbnb, has experienced meteoric...
Countrywide, two years after Platt - time for the last rites?
In six years of editing Estate Agent Today, some of my most dramatic moments have...
It's time to sort the wheat from the chaff when it comes to overseas buyers of property.
If you listen to Theresa May you'd think that by getting foreign purchasers to pay a stamp duty surcharge you would (hey presto!) solve the UK's homelessness problem.
A few days after May's comments, some joint research from Channel 4 News and the Bureau of Investigative Journalism revealed for the first time a hard and fast figure for the number of homeless who had died in the UK in the past year.
It was a shocking 449, many of them quite literally dying on our streets: we didn't hear any comment about the scandal of homelessness from Theresa May when that story broke. Her interest in the issue appears to have waned since she survived the Tory conference.
So what are we to make of her overseas buyer stamp duty? The surcharge will of course further bolster Treasury coffers but nowhere near enough to 'solve' homelessness.
Instead, my fear is that this gesture will put up another small 'UK not open for business' sign, another decibel or two on Boris Johnson's "F*** business" mood music that so many of May's leadership rivals have been happy to play recently.
And as for the current London housing market, there are obvious dangers in discouraging foreign buyers: look no further than Foxtons as evidence of the capital's challenges.
But whilst this latest surcharge looks like a bone for Brexiteers (with our industry suffering as a consequence) let's look at a more genuine problem with overseas ownership.
And this is one that government could help remove rapidly, simultaneously reducing overseas ownership and improving the UK's standing in the global community.
I mean, of course, that the government could clamp down on the dubious practice of overseas company ownership of property the so-called 'dirty money' which casts a stain over many aspects of the high-end agency and residential development sectors in London.
Earlier this year it was revealed that 50,000 properties in London were held by firms registered abroad, worth £33.9 billion according to the BBC/Land Registry research. In Kensington & Chelsea alone there are 6,000 such properties.
Now we can play coy if we want to, but in reality we know the issues behind the figures.
The National Crime Agency has estimated that £90 billion of criminal funds are laundered through the UK each year, with the London property sector a likely means.
Many of the properties already identified by the BBC are owned by companies in the British Virgin Islands, a Caribbean cluster with 31,000 residents who own 23,000 UK properties: oh yes, there's concern over the BVI's tax transparency, too.
That reflects badly on the UK and London far more badly than some foreign investor buying a flat in Canary Wharf and now being targeted by the stamp duty surcharge.
This dirty money problem is not confined to the capital. That BBC analysis found that in the south east there were £7.2 billion of properties owned by firms registered abroad; the West Midlands and North West have £3.1 billion apiece.
Some legislation is on its way: a draft Bill published by government says overseas companies will have to reveal their true beneficial owners and the first Unexplained Wealth Order, obliging someone to reveal their source of wealth, has already been served.
These are steps in the right direction, and let's hope there are resources at Companies House and elsewhere to identify suspicious ownership on a far wider scale. If that could happen it would both improve British prestige and free some homes for other buyers.
In the meantime, though, let's not pretend the biggest problem is the relatively modest number of wealthy families who are completely legally buying homes in pockets of London.
Such wealthy families may have little need for our sympathy, but let's not pretend that forcing them to pay extra stamp duty does anything to solve our housing crisis - or to make us look open for business just when we could easily appear inward and isolated.
*Editor of Estate Agent Today and Letting Agent Today, Graham can be found tweeting all things property @PropertyJourn
Graham Norwood
Overseas Property Ownership
13 October 2018 10:05 AM
Interesting hyphen-ation!
MIRAS: Could Theresa May go back to the future? ...
I'm indebted to one of the most enthusiastic estate agents I've...
What would a Corbyn government mean for sales and lettings? ...
While media attention has been on Theresa May, Brexit and the...
Forget stamp duty reform - There's a new tax on the horizon ...
When it comes to provoking anger in property professionals, taxation takes...
So farewell Theresa May and thanks for...well, not much ...
No, this isn't about Brexit. But as the saga of leaving...


Negative content on the Internet might damage your reputation. Remove it with LinKiller.
LinKiller monitors the Web, locates negative content regarding companies and individuals and eliminates them from the Net.
We share information on the Internet, but we cannot always remain in control of it. In addition, third parties may upload content that concerns us (images, videos, etc.) or post it on our behalf (newspaper articles, posts, comments). Here is the purpose of LinKiller: to find all content that undermines the reputation and eliminate or de-index it from the Web.
LinKiller is immediate and fast: an app that can be accessed anywhere, anytime.
You can check in real time the negative content, the number of deleted links, the progress of the removal and de-indexing process and the sources where the content has been published.
How do I get the Web App?
We will provide you with the credentials to register and start your rebirth.


The news that commuters using New York City Metropolitan Transportation Authority (MTA) subways and buses will soon be able to "tap a contactless bank card or their mobile wallet at turnstiles" inspired CNBC to declare that "a world without cash is coming soon."
We're seeing financial services brands leverage major sponsorships in pursuit of specific business objectives. For example, Visa, Amex and Mastercard recently activated their sponsorships of premium events to increase trial and use of contactless payments.
With marketing that is consistent, compelling and relatable – particularly among its Millennial cardmembers– Amex brings high focus and a big investment to "Pay It Plan It," which allows Amex cardmembers to designate purchases of all sizes for payment in new ways and with greater control.
Brands on Instagram face a pretty big challenge: how to create content that achieves the level of authenticity platform users expect. One approach is to leverage the audiences of popular content creators, as in these recent posts published in partnership with top FIs, including Amex, Bank of America, Capital One, Chase, U.S. Bank and Visa.
Financial institutions are becoming more willing to creatively explore popular topics & platforms in their marketing strategies.
New marketing shows Amex has a strategy in place to improve acceptance stats and perceptions.
These examples from Chase, American Express and Citibank demonstrate how financial services brands can use Instagram to support both their value proposition and the platform's mission.
Although we really think the tone/message is spot-on, the campaign's execution on Facebook and Instagram (@americanexpress ) is where it has the potential to get really interesting.
Amex is betting on BuzzFeed's winning formula for attracting and engaging Millennial consumers, taking advantage of a tremendous opportunity to get in front of millions (and millions) of their eyeballs via some of the Internet's most popular videos… before anyone else does.


Since 1992, Mathis Exterminating has been offering outstanding extermination and pest control services. An expert team of extermination professionals, led by founder Damon Martin, have served 50,000 happy customers.
Mathis Exterminating prides itself in being an industry leader through the use of eco-friendly pest control methods, as well as their innovative and advanced techniques in pest control. Our primary concern at Mathis Exterminating is serving our customers' needs. In order to better serve our customers, we are constantly searching for updated pest extermination methods, products, and tools. We have been in the pest control industry for nearly 20 years, and guarantee our services. If there is a problem, you can trust that it will be resolved.
Expertly trained by certified pest control professionals, all technicians undergo continuing education in the field. This ensures that they are updated on all of the latest techniques and methods of pest control.
Each technician is a well-groomed professional. Not only are they punctual, but they are clean and friendly as well.
Each technician will arrive in a safe and clean vehicle.
Each technician is certified by the state.
Within 24 hours of being notified, you will receive a response from a certified technician.
With more than 49 years of collective experience, Mathis Exterminating dominates the pest control industry.
Mathis Exterminating uses the most current pest treatment methods.
Throughout the spring and summer months, Mathis Exterminating is available to serve you on Saturdays. We also have a messaging service available 24 hours a day.
Protect your Family & Home!


When Jimmy K's first opened it's doors in 2000, our signature sandwiches quickly created a name for themselves, FAMOUS!! From our Wezzee to the Chicken Pita, our sandwiches are Outrageously Delicious!! We pride ourselves on serving the best food and giving the best service!
Staff is very sweet and eager to please. Food was made to order, but service was fast! We had hot dogs, a burger and a buffalo chicken wrap. All were delicious... Will return!
Best gyro I've eaten in a long time ,, great food at a great price!
...The Gyros were fantastic, and the service was pretty good. The staff was nice and friendly. I would go back here again next time I am in town.
This is a little place specializing in gyros, italian beefs, dogs, etc..... The gyros were great, good sauce, too!
I had a chicken pita and it was delicious, but my husband got a gyro and I can say it is absolutely the best I have ever had!! Amazing food!!


text:
Q: Exclude left join in case of empty cells So I need to join the company table only in the case that the email field is populated.
Is there a way to exclude the company table join in case there is an empty email field?
select * from person p left join company c on(substring_index(p.email, '@', -1)=c.website and p.email <> '')

I can't do it with a "where" clause because I am joining yet another table, and I want that table joined regardless.
Any help would be appreciated, thanks!

A: I figured it out! A flash of inspiration, and...
select * from person p left join company c on((case when length(p.email)>0 then substring_index(p.email,'@',-1) else "purplepower" end)=c.website)

By using the "case" clause I have made sure that only the email values that exist are matched.  If not, it is matching an arbitrary string which of course doesn't match to the company table.

A: That's overly complicated, and an elaborate workaround that is unnecessary. By nature a LEFT JOIN will attempt to join the other table on the column you specify and if it cannot join it, columns from that table are NULL. Also, a join is used to return columns from multiple tables based on a joining condition. Since you are using a left join, you will return all rows from the person table regardless of the email value.
Since you don't select anything from company and use a left join, your workaround solution query is the same as:
select * from person p
where p.email <> ''

Based on your comment that there is another table in the works, you could force company to have a value:
select * from person p
left join company c on (
    substring_index(p.email, '@', -1) = c.website
    AND c.website <> '')

A: select * from (select * from person where email <> '') p
  left join company c
  on(substring_index(p.email, '@', -1)=c.website )

redpajama_set_name: RedPajamaStackExchange


text:
The Curious Case Of Sharbat Gula – The Green Eyed Afghan Girl
By crssblog December 1, 2016 January 23, 2017
Leave a Comment on The Curious Case Of Sharbat Gula – The Green Eyed Afghan Girl
Pakistan recently faced a PR nightmare when Sharbat Gula – the once famous National Geographic Afghan girl – was deported to her native Afghanistan. Gula was accused of forgery, living in Pakistan using fake identity, and thus was not only detained, but was later sent back to Afghanistan.
Gula first gained international prominence in 1984 as an Afghan refugee girl, after Steve McCurry's photograph of her – with striking and unique green eyes, was published on the cover of National Geographic. Her portrait, as National Geographic claims, became a symbol for refugees all over the world. She then resurfaced in 2014, but accusations of forging her identity documents sent her back into hiding.
This incident too turned international media attention on Pakistan, and yet again, for all the wrong reasons. The western media cried foul over the apparent "injustice" with Sharbat Gula and her family, yet there was rarely a mention of the illegitimacy" of Gula's stay in Pakistan.
What the western media also ignored was that most Afghan refugees returning home are facing financial and residential struggles. Whereas, Gula not only visited the Presidential Palace and had a one on one meeting with President Ashraf Ghani, but was also fortunate to receive a furnished apartment from the Afghan government.
Even though this seemed more like a PR exercise by Kabul, none of the media outlets questioned Ghani and his government over the adverse circumstances faced by the rest of the refugees. Afghan media's reaction to this story was nothing short of displaying Pakistan in poor light internationally, without acknowledging that hundreds of thousands of Afghan families are not only legally based in Pakistan, but also earn their livelihoods.
Soon after Gula landed in Afghanistan, the narrative among the general Afghan population and the Afghan media was that of "Gula treated with disgrace in Pakistan". Yet, many Afghans were unaware that both the provincial government of Khyber Pakhtunkhwa, and federal government in Islamabad agreed to help Gula and challenged the court's decision against her deportation. Also, the narrative in Afghanistan also excluded the fact that many Afghans, holding legitimate proof of ID, are living in Pakistan without facing the threat of deportation.
Gula's story was also another case of "larger than life" storylines in the western media about the misery of someone only when they needed them. A similar example was that of the Syrian boy, Omran, sitting mortified in an ambulance after losing his family in the ongoing Syrian conflict. Before or after that story, there were rarely any mentions of the misery of Syrians in any prominent media outlets with such a magnitude or scale. Gula's case is also somewhat similar, as before her deportation, none of these media outlets published stories, or made efforts to highlight the plight she might be living in. Additionally, in the month of November alone, Turkey, Iraq, South Sudan, and Syria have had more than a dozen deadly terror attacks, killing scores; yet these stories could only make it to the back pages of all media networks as the US Presidential election enjoyed a higher coverage in comparison.
Various human rights groups joined the chorus with the western media against the "grave injustice" committed by the Pakistani authorities against Gula. Champa Patel, Director South Asia of Amnesty International, voiced her discontent by arguing that Gula's deportation would tarnish Pakistan's status of a generous host. Yet, her statement disregards over two decades of efforts that Pakistan has put in to accommodate a fair share of 3 million Afghan refugees. A single case, based on media profile and hype, tarnishing a country's image sounds less than rational.
It would also be unfair to criticise Gula in this whole episode as her only aim was to survive as a widow, while raising her four children. Yet, she committed a procedural mistake by not availing the first Afghan refugee census, conducted in 2006, which could have enabled her to stay in Pakistan legally.
Yes, Pakistan might (and should) have created an exception for Sharbat Gula. Yes, Gula's case demanded more compassion than reason. And yes, apparent mishandling of Gula's case might have brought shame to Pakistan in the western media circles. But the story, which gained global attention, should also generate debates on how western media picks and chooses people, groups, and topics according to their need and appetite, and ignore their miseries when the stories might not contain "buzzworthy" appeal.
The author, Farooq Yousaf, is a Research Fellow and a Programme Consultant at CRSS. He is currently pursuing his PhD in Australia.
Tags: Farooq Yousaf
Previous Entry Still Weighing Options
Next Entry Newswise – December 05, 2016


text:
Biografia
Nato a Perth nell'Australia occidentale, figlio di Ernest e Isobel Ferguson, Cuthbertson si esibì, sin dalla tenera età, in teatro e in radio.

Durante la seconda guerra mondiale prestò servizio come tenente di volo con la RAAF, dal 6 dicembre 1941 al 1 luglio 1947, compreso il servizio con il 111° Air Sea Rescue Flight.

Cuthbertson arrivò in Gran Bretagna nel 1947 e poco dopo apparve in Romeo e Giulietta ai Boltons, nel ruolo di Romeo. Nel West End di Londra apparve nei panni di Laerte in Amleto, Aimwell in The Beaux Stratagem e di Octavius Robinson in Uomo e superuomo.

Sul grande schermo interpretò spesso ruoli di militare: fu il colonnello Henniker in Per una questione di principio (1954), accanto a David Niven, il capitano Eric Simpson in Whisky e gloria (1960), e il maggiore Baker in I cannoni di Navarone (1961), nuovamente al fianco di Niven. Fece anche una breve apparizione come ufficiale di stato maggiore in Birra ghiacciata ad Alessandria (1958). Nel 1962 interpretò un insegnante di scuola in L'anno crudele con Laurence Olivier.

Apparve quattro volte nella serie televisiva Agente speciale.

Cuthbertson aveva anche un talento per la commedia che lo portò al suo ruolo più noto, il colonnello Hall nell'episodio Il gourmet di Fawlty Towers della sitcom di successo Fawlty Towers (1975). Apparve in altre occasioni alla televisione britannica insieme a Tommy Cooper, Dick Emery e Frankie Howard in All Gas and Gaiters, Gioco pericoloso, UFO (nell'episodio Il triangolo quadrato) e Terry and June, e fu ospite regolare al The Morecambe & Wise Show dal 1973 al 1976.

Uno dei suoi ultimi ruoli sul palcoscenico fu in The Corn Is Green di Emlyn Williams all'Old Vic nel 1985.

Vita privata
Cuthbertson è stato a lungo sposato con la dottoressa Gertrude Willner, una rifugiata della Cecoslovacchia occupata dai nazisti, che in origine era stata un avvocato ma divenne poi insegnante in Gran Bretagna.

I due avevano un figlio adottivo.

Al momento della sua morte Cuthbertson viveva a Surbiton, nel Surrey.

Filmografia

Cinema
 Per una questione di principio (Carrington V.C.), regia di Anthony Asquith (1954)
 Il segno del pericolo (Portrait of Alison), regia di Guy Green (1955)
 On Such a Night, regia di Anthony Asquith  (1956)
 Doublecross, regia di Anthony Squire (1956)
 Cloak Without Dagger, regia di Joseph Sterling (1956)
 L'uomo che non è mai esistito (The Man Who Never Was), regia di Ronald Neame (1956)
 Donna da uccidere (Eyewitness), regia di Muriel Box (1956) - non accreditato
 Anastasia, regia di Anatole Litvak (1956) - non accreditato
 The Passionate Stranger, regia di Muriel Box (1957)
 Fuoco sullo Yangtse (Yangtse Incident: The Story of H.M.S. Amethyst), regia di Michael Anderson (1957)
 Il capitano soffre il mare (Barnacle Bill), regia di Charles Frend (1957)
 Benvenuto a Scotland Yard! (Law and Disorder), regia di Charles Crichton (1958)
 Birra ghiacciata ad Alessandria (Ice Cold in Alex), regia di J. Lee Thompson (1958)
 La battaglia segreta di Montgomery (I Was Monty's Double), regia di John Guillermin (1958)
 La strada dei quartieri alti (Room at the Top), regia di Jack Clayton (1959)
 The Crowning Touch, regia di David Eady (1959)
 Il fronte della violenza (Shake Hands with the Devil), regia di Michael Anderson (1959)
 Il discepolo del diavolo (The Devil's Disciple), regia di Alexander Mackendrick e Guy Hamilton (1959)
 Ombre sul Kilimanjaro (Killers of Kilimanjaro), regia di Richard Thorpe (1959)
 Frontiera a Nord-Ovest (North West Frontier), regia di J. Lee Thompson (1959)
 Gli strangolatori di Bombay (The Stranglers of Bombay), regia di Terence Fisher (1960)
 Whisky e gloria (Tunes of Glory), regia di Ronald Neame (1960)
 The Malpas Mystery, regia di Sidney Hayers (1960)
 Man at the Carlton Tower, regia di Robert Tronson (1961)
 I cannoni di Navarone (The Guns of Navarone), regia di J. Lee Thompson (1961)
 Un generale e mezzo (On the Double), regia di Melville Shavelson (1961)
 Solo for Sparrow, regia di Gordon Flemyng (1962)
 L'anno crudele (Term of Trial), regia di Peter Glenville (1962)
 The Boys, regia di Sidney J. Furie (1962)
 L'uomo che vinse la morte (The Brain), regia di Freddie Francis (1962) - non accreditato
 La signora sprint (The Fast Lady), regia di Ken Annakin (1962)
 Freud - Passioni segrete (Freud: The Secret Passion), regia di John Huston (1962)
 9 ore per Rama (Nine Hours to Rama), regia di Mark Robson (1963)
 Un buon prezzo per morire (The Running Man), regia di Carol Reed (1963)
 Mani sulla luna (The Mouse on the Moon), regia di Richard Lester (1963)
 Tamahine, regia di Philip Leacock (1963)
 Bitter Harvest, regia di Peter Graham Scott (1963)
 Doppio gioco a Scotland Yard (The Informers), regia di Ken Annakin (1963)
 La settima alba (The 7th Dawn), regia di Lewis Gilbert (1964)
 Operazione Crossbow (Operation Crossbow), regia di Michael Anderson (1965)
 Flagrante adulterio (Life at the Top), regia di Ted Kotcheff (1965)
 Game for Three Losers, regia di Gerry O'Hara (1965)
 Combattenti della notte (Cast a Giant Shadow), regia di Melville Shavelson (1966)
 Press for Time, regia di Robert Asher (1966)
 La grande sfida a Scotland Yard (The Trygon Factor), regia di Cyril Frankel (1966)
 Quei fantastici pazzi volanti (Jules Verne's Rocket to the Moon), regia di Don Sharp (1967)
 Half a Sixpence, regia di George Sidney (1967)
 La forca può attendere (John Huston), regia di John Huston (1969)
 Galaxy Horror (The Body Stealers), regia di Gerry Levy (1969)
 Il capitano Nemo e la città sommersa (Captain Nemo and the Underwater City), regia di James Hill (1969)
 L'ultimo avventuriero (The Adventurers), regia di Lewis Gilbert (1969)
 Controfigura per un delitto (One More Time), regia di Jerry Lewis (1970)
 Sadismo (Performance), regia di Nicolas Roeg e Donald Cammell (1970)
 The Firechasers, regia di Sidney Hayers (1971)
 Assault, regia di Sidney Hayers (1971)
 Diamonds on Wheels, regia di Jerome Courtland (1974)
 The Outsider, regia di Tony Luraschi (1979)
 L'oca selvaggia colpisce ancora (The Sea Wolves), regia di Andrew V. McLaglen (1980)
 2 sotto il divano (Hopscotch), regia di Ronald Neame (1980)
 Assassinio allo specchio (The Mirror Crack'd), regia di Guy Hamilton (1980)
 Agatha Christie: 13 a tavola (Thirteen at Dinner), regia di Lou Antonio (1985)

Televisione
 Robin Hood (The Adventures of Robin Hood) – serie TV, episodio 2x27 (1957)

Collegamenti esterni

redpajama_set_name: RedPajamaWikipedia


Are you an experienced shopper marketing or category manager, with demonstrable experience in an eRetail or ecommerce role? Would you like to play a pivotal role in ensuring uptake of shopper marketing programmes within the ever evolving consumer technology space? If yes, then please read on!
A leading global business in consumer technologies (mobile devices, tablets, wearables, digital appliances…) has opened a brand new role to support both in life and new customer acquisition within their online retail channels. As such they are looking for a talented online marketing professional with proven experience from within a retail setting.
-Pioneering customer experience initiative and implementing a net promoter system.
-Transforming relationships with strategic international partners to collaborate on shopper programs.
-Spearheading category drivers and owning account strategy internationally.
-Identifying omni-channel category or shopper opportunities in a Pan-European setting.
-Tracking and reporting on category retailer performance.
-Optimising the "digital shelf" to enhance the etailer shopping experience.
-Enhancing traffic and conversion through online assets.
-Developing etailer toolkit and guidelines by account and channel type.
-Forecasting the impact of category drivers and measuring pilot store impact.
-Demonstrating excellent, omni-channel experience and an expert understanding of digital retail initiatives.
-10+ years' experience of working in a high-performing retail sales and marketing team in an online or ecommerce category planning, shopper or merchandising role.
-Proven experience as a category manager or similar.
-A great understanding of how to apply category insight and strategy to drive sales.
-An understanding of data analysis and forecasting methods.
-Experience in working with 3rd part vendors.
-Outstanding client and stakeholder relationship management skills.
-Strong ability to translate numbers into impactful messaging.
-Able to present to senior executives with flair and confidence.
-Must have experience of an online merchandising role at operator retailer level within FMCG.
-A genuine passion for technology.


STUDENTS OR PROFESSIONALS THIS ONE BEDROOM HOUSE IN EXCELLENT CONDITION!!. laminate flooring. Modern kitchen with integral fridge-freezer, cooker, washer/dryer. The apartment comes furnished with the sofa, a bed and a wardrobe. Modern bathroom with bath and shower. Secure entry system.
CLOSE TO CITY CENTER AND COVENTRY UNIVERSITY!!!! Ground floor flat benefits 3 bedrooms Modern large kitchen Lounge Shower room Fully furnished.


1: Find the Best Light in your House.
Sometimes life happens, and we need to capture those moments when we aren't in the ideal room with the ideal light. I get it. If that is the case, open the curtains/blinds and let as much light in as possible.
2: Clear the Clutter and Lower your Aperture.
I have two toddlers, clutter is my life. But you can tell when I've been shooting inside because then my house is usually somewhat tidy. I joke that I wish my house was as clean as it looks in photos. Big toy bins are my saving grace, grab all the 'stuff' and dump it in the bin and close the lid, out of sight out of frame. Heck, you don't even need to go that far, just throw it out of the frame, behind you in a pile if need be. You want to showcase your subject, not the hairbrush on the floor. Why have something in your photo that is going to draw your viewer's eye away from the focus on your subject? Lowering your aperture is another great way to bring the focus to your subject and off of your house in the background, just turn that clutter into beautiful bokeh.
You certainly do not need to be a professional to critic your own work. Look at a photo you recently took and analyze it. Are you using any rules of composition? if so which ones and why? what is the mood of the photo? Is it properly exposed? Do you like the use of light? What don't you like about it? Is there something distracting?
By looking at your work and determining what you like and don't like, you slowly start to figure out your style. As parents, we often will like an image because our child looks adorable in it. Some photographers say that is using "Mommy goggles". That is perfectly fine. You should love the images of your kiddos even if they aren't technically perfect. But if you want to get better as a photographer this is great practice, I do this all the time!


For the third time, H. E. Rinchen Dorjee Rinpoche presided at "The Great Unlimited Amitabha Puja for Transferring Consciousness" at the Taipei Arena on September 9, 2007 (Sunday). 12,000 people, including the monks and nuns from both esoteric and exoteric Buddhism, the mantrikas, the khenpos, the lamas and many from different corners of the world, attended this great and precious puja. Upon completion of the puja, in an effort to provide funding for social welfare and assistance for the needed, Rinpoche donated NT$ 4,896,905, which inculded all the contributions received from the believers and his personal offering, to the Ministry of Interior.
The puja was attended by over 12,000 believers, including the monks and nuns from both esoteric and exoteric Buddhism, the mantrikas, the khenpos, the lamas and many from different corners of the world.
Rinchen Dorjee Rinpoche conducting the practice with solemnity.
Rinpoche introduced Buddhism with his vast wisdom and comprehensive speech, unveiling the greatness and ingenuity of the Buddhist doctrine to the general audience.
The puja was perfectly completed.
The believers receiving blessed water after the puja.
Two days before the great puja, which had been planned for over a year, lots of volunteers from the Glorious Jewel Buddhist Center were devoted to the on-site preparations.
Volunteers making tormas with undivided attention.
Prior to the 2007 great unlimited Amitabha puja for transferring consciousness, the red carpet was covered with a cloth printed with the 8 auspicious symbols for the entrance of H.E. Rinchen Dorjee Rinpoche.
Rinpoche entering the venue, preceded by the procession of the 8 auspicious symbols, the sutra banners and the dharma instruments,.
Rinchen Dorjee Rinpoche lighting a butter lamp to offer to Buddhas.
Selected at random, one thousand attendees took turns lighting the lamps on behalf of all.
Those offering mandalas were selected at random as well, through a lucky draw held upon their entry to the venue.


To simplify navigation on the BizTalk articles, I've selected only the best articles.
BizTalk Server and Agile. Can they live together?
Complex XML schemas. How to simplify?
I've decided to keep live the first version of this post. It is interesting how tests evolve and how results change. V.1.0.10 see here.
The v.2 was inspired by OniBait, thanks a lot! He added code for Avro and Bond serializers, added 99st% to measurements, made other improvements which significantly impacted results.
That's the force and beauty of the open source.
Small improvements in the project code change results all the time. So I just mention the last improvement date.
The project code is on GitHub.
Any distributed system requires serializing to transfer data between systems and applications. The serializers used to be hidden in adapters and proxies, where developers did not deal with the serialization process explicitly. The WCF serialization is an example, when all we need to know is where to place the [Serializable] attributes. Contemporary tendencies bring serializers to the surface. In Windows .NET development, it might have started when James Newton-King created the Json.Net serializer and even Microsoft officially declared it the recommended serializer for .NET.
There are many kinds of serializers; they produce very compact data very fast. There are serializers for messaging, for data stores, for marshaling objects.
What is the best serializer in .NET?
Nope… this project is not about the best serializer. Here I show in several lines of code, how to use different .NET serializers. Want to serialize object and look for the sample code? You are on the right place, just copy-past this code in your project. The goal is to help developers with samples. Samples should be simple enough to copy-past code without effort. Also samples should provide effective code to be good enough for most messaging scenarios. I want to show serializer in the simplest way, but it is good to know that it would not hit your code performance. That is why I added some measurements, so you can do right decisions.
Please, do not take these measurements too seriously. I have some numbers, but this project is not the right place to get decisions about serializer performance. I did not spent time to get the best results. If you have the expertise, please, feel free to modify code to get numbers that are more reliable.
Note: I have not tested the serializers that require IDL for serialization: Thrift, , Cap'n Proto, FlatBuffers, Simple Binary Encoding. Those sophisticated beasts are not easy in work; they needed for something more special than straightforward serialization for messaging. These serializers are on my Todo list. ProtoBuf for .NET implementation was upgraded to use attributes instead of IDL, kudos to Marc Gravell. Single exception is the new Microsoft Bond (thank OniBait again for coding Bond part).
Most of serializers installed with NuGet package. Look to the "packages.config" file to get a name of the package. I have included comments in the code about it.
The test data created by Randomizer. It fills in fields of the Person object with randomly generated data. This object used for a single test cycle with all serializers, and then it is regenerated for the next cycle.
If you want to test serializers for different object size or for different primitive types, change the Person object.
The measured time is for the combined serialization and deserialization operations of the same object. When serializer called the first time, it runs the longest time. This longest time span is important and it is measured. It is the Max time. If we need only single serialization/deserialization step, this is the most significant value for us. If we repeat serialization / deserialization many times, the most significant values for us are Average time and Min time.
The Avg-100% : all measured times are used in calculation.
The Avg-90% : 10% slowest measurements ignored.
Some serializers serialize to strings, others – to the byte arrays. I used a string as a denominator. The base64 format is used to convert byte arrays to strings. I know, it is not fair, because in many cases we use only a byte array to serialize, not a string. UTF-8 also could be more compact format.
Again, do not take test results too seriously. I have some numbers, but this project is not the right place to get conclusions about serializer performance. You have to take this code and run it on your specific data on your specific workflows.
Here is a test result for the 100 repetitions. This number of repetitions shows stable results.
There is no such thing as the "best serializer". If you invest time in optimizing code, the loser will be winner. If you change the test data, the winner would not be winner anymore. So I will mark serializer as a "winner" if it shows significant leadership in numbers, not just 1-3%.
There are several winners in each category.
This category important if you need smallest data size (on wire, in store…). Look at the Size: Avg measurement. All winners in this category use proprietary unreadable formats.
All Json serializers create strings with almost the same size.
You can see the output serialized strings. They written to the trace output, so use DebugViw to see them.
Many Json serializers do not work well with DateTime format out-of-box. Only NetSerializer and Json.Net take care of DateTime format without additional customization.
This category important, if you only need make a single serialization/deserialization. Look at the Max measurement.
Avro, Bond, and NetSerializer also show good results.
Jil shows the worst result but, again, it means nothing.
Serializers show biggest incoherence in this category.
This category important, if you need many serialization/deserialization acts on the same objects. Look at the Min and Avg:-90% measurements.
MsgPack and BondJson also show good results.
This category important, if you need several serialization/deserialization acts on the same objects. Look at the Avg:-100% measurement. It is a mix of slowest and fastest times. As you can see, MessageShark, Jil and ProtoBuf not in leaders anymore because of the suboptimal initialization. But Json.Net now is a leader because of good initialization time on our test data.
The classic Json.Net serializer used in two Json.Net (Helper) and Json.Net (Stream) tests. Tests show the difference between using the streams and the serializer helper classes. Helper classes could seriously decrease performance. Therefore, streams are the good way to keep up to the fast speed without too much hassle.
The first call to serializer initializes the serializer, that is why it might take thousand times faster to the following calls.
For Microsoft Avro my simple serialization interface is patched. For some reason it is not possible to pass generic T type into it. Type was hardcoded. Because of this hack, Avro makes tests faster.
Json and Binary formats bring not too much difference in compacting the serialized strings.
Serialization of the classes with explicit constructors usually has negative impact on the serialization speed. That's why test class (Person) creation is implemented not by constructor but by a method.
Many serializers do not work well with Json DateTime format out-of-box. Only NetSerializer and Json.Net take care of DateTime format without special treatment.
Test prints the test results on the console. It also traces the errors and the serialized strings, which can be seen in DebugView for example.
The Apolyton.Json and HaveBoxJSON serializers failed in tests, that why we see zeroes in several measurements.
XML Transformation: Xslt vs .NET. Part 3.
Max Toro commented on the Part 1 and his suggestions changed a lot.
So, I have added this Part. It shows, how a small optimization could change the test results.
Actually, I have added just two lines of code, that's it.
Measurements got additional numbers. The Max time is important in cases, where we want to do a single transformation. Min and Average-90% times are important, when we need a lot of cycles transforming the same objects. Average-100% time is important, when we are somewhere in between. 99st% shows how fast the times are stabilized and moved towards the Min.
Xslt transformation speed is on a par with Object transformation speed on broad spectrum of the message sizes.
If you use Json instead of Xml, Json is always faster. The object size is matters.
The object size is more important than the method of transformation for large objects. So the best way to improve speed for the large objects is to compact the object serialized string. Remove all indents, spaces, all white space and this significantly speeds up the transformation. Remove it for both, the source and the target strings.
The speed differences between three mentioned transformation methods are not significant, so choose on base of the development skills and speed. The development skills and knowledge beats the technology.
In the first part, I have compared the Xslt transformations of the XML documents with the Object transformations. It happens the Object transformations are faster in most cases. They are also simpler from the developer point of view.
If we got a freedom to choose Object transformations instead of Xslt transformations, it might be, we got even more freedom and choose Json or other contemporary serializing formats instead of XML. If this is a case, we should know possible improvements, right?
The test project code is on the GitHub.
I have added NetSerializer Json serializer. You can get it as a NuGet "NetSerializer" package. It is one of the fastest Json serializer in .NET. The Object transformations now executed in two modes: with XML serialization and with Json serialization.
XML Transformation: Xslt vs .NET. Part 1.
The Xml transformation is an important part of the system integration. The Xml documents are everywhere despite surging JSON.
When we need to transform [or map] one Xml document to another Xml, we have several options. Two of them prevail. The first is the Xslt language. The second is the object transformation.
In reality to make the Xslt map we possibly need the XML Schema for source and target Xml documents. The XML Schemas are not mandatory but many Xslt editors use them to create Xslt maps. The BizTalk Server Mapper is such example.
The Xslt map is defined as the Xml document itself. The Xslt operators and expressions defined with help of the XPath, another Xml related standard.
An Object transformation is a transformation, when Xml is converted to the object graph of any programming language like Java, or C#. Then objects mapped to other objects, which converted back to Xml. Those two conversions could be performed by XmlSerializer which is part of .NET. The mapping written and executed as the generic C# code.
The Object transformation is not an official term. Usually in development you see terms "mapping", "transformation", "converting".
Theoretically, the more specialized tools should always beat the less specialized. The Xslt designed for exactly this purpose, so the question is how better is it? Transformation speed is the most important feature, so I tested the speed.
Tests have two parameters: the number of repetitions and the number of the nested objects.
The repetitions should stabilize the test measurements, make them statistically more correct. Number of the nested objects implicitly defines the size of the transformed Xml document.
The test Xml document is created by XmlSerializer from the Person class.
The XsltTransformer uses the PersonToEmployee.xsl and PersonToPerson.xsl Xslt stylesheets. If you want to use the third-party mappers to create or change the stylesheets, I have generated Xml schemas for you; they are in the XmlSchema.xsd file.
NetTransformer uses the same XmlSerializer. The transformation code is simple and boring, nothing to say about it.
The transformers do not try to produce the same transformations. Small differences in transformations don't matter for our case.
Enrichment, when the source and target Xml documents have the same schemas. It used to change content of the documents without changing the document structure.
Transformation, when the source and target Xml documents have different schemas. It creates a new target document using data of the source document. Target document has a schema, different from the source document schema.
For enrichment we operate with the same document structure, so theoretically enrichment should be simpler and faster.
The test document created and tested for Xslt and Object transformations, and for both Enrichment and Transformation types. It is one test cycle.
A new test document created for each test cycle. This eliminates the possibility of the optimizations, which could be performed on the OS, the memory management level. The data for the test document created randomly by Randomizer class.
The test classes are initialized on the first cycle, so it takes much more time to perform the first test. I measure the maximum time, because it is important for the real life situation, when we need only one transformation.
To measure the average time the 5% of maximum and 5% of minimum values are removed from calculation.
I also measure the size of the transformed Xml document. It shows the importance of the spaces and new line symbols for the document size.
The sizes of the result Xml documents for Xslt and Object transformations should not be very different. Yes, you read me right. The result Xml documents should not be the same in each symbol for both transformations, and still documents can be recognized as equal. It is because of the ambiguity of the Xml standard. For example, the namespace prefixes could be different for the same namespace. In one result we can get "ns0:" prefix and the "abc12:" prefix in another, but both resolve the same namespace. As result the Xml documents got different size, but both are equal in terms of data values and structure. As result of this, we could not compare the Xml documents as the strings. We could converted Xml documents to the objects graph and compared the result object set. If all objects are equal, Xml documents are equal. I decided do not compare results because it is not the test goal. I just output the target Xml documents, so they could be easily compared, if needed.
The Object transformation won in both Transformation and Enrichment tests.
For this test the size of the Xml document was about 10K. I have change the size of the tested documents. When size grows, the difference between Xslt and Object transformations start to decrease. The average times were almost equal for the documents with 1M size.
It worth to mention the unstable result times. I write all measured times into Trace output.
As you can see, the measured times are stable but sometimes they grow significantly. Possibly, it is result of the garbage collection. Please, take this into account.
Xslt transformation is not faster than Object transformation, at last for the Xml documents smaller than 1M.
Object transformation significantly faster for Xml documents smaller than 100K.
Results are unexpected. Aren't we the MythBusters :) ?
Is it worth to use Xslt, if we have a choice?
To use Xslt we have to study a new language, the Xslt. We have to study XPath, XML and maybe XML Schema standards. All this require time and effort. There are the special skills for testing and debugging Xslt code.
The Object transformation created in pure, generic programming languages. There is almost zero new knowledge we need to do it.
What about time spent on the creating transformations? There is not too much good Xslt mappers on market. The BizTalk Server Mapper one of the best. It is very performant in simple cases. But if there is a little bit more complexity, the mapper is not booster anymore but stopper. There are several books with tips and tricks how to do things with this Mapper. Are there such books for the Object mapping? Not sure they are, because the Object mapping is just generic programming, ABSOLUTELY NOTHING SPECIAL.
So, the conclusion is: use Xslt only for very, very, very good reason. Is it not faster at execution, It might be faster but without significant margin. It is not faster at development in most cases, especially if we claim all the time we spent on studying. Any programmer could develop Object transformations and only skilled Xslt developer could develop Xslt transformations.
Feel free to upgrade source code on GitHub. Please, report any issues and propose any ideas here.
I've added the tests for very small and very big XML documents. Results fit the above conclusions.
Please go to the v.2 of this article.
Any distributed system requires serializing to transfer data through the wires. The serializers used to be hidden in adapters and proxies, where developers did not deal with the serialization process explicitly. The WCF serialization is an example, when all we need to know is where to place the [Serializable] attributes. Contemporary tendencies bring serializers to the surface. In Windows .NET development, it might have started when James Newton-King created the Json.Net serializer and even Microsoft officially declared it the recommended serializer for .NET.
No, no, no, this project is not about the best serializer. Here I gather the code which shows in several lines of code, how to use different .NET serializers. Just copy-past code in your project. That is the goal. I want to use serializer in the simplest way but it good to know if this way would really hit your code performance. That is why I added some measurements, as the byproduct.
Note: I have not tested the serializers that require IDL for serialization: Thrift, new Microsoft Bond, Cap'n Proto, FlatBuffers, Simple Binary Encoding. Those sophisticated beasts are not easy in work, they needed for something more special than straightforward serialization for messaging. These serializers are on my Todo list. ProtoBuf for .NET implementation was upgraded to use attributes instead of IDL, kudos to Marc Gravell.
The test data created by Randomizer. It fills in fields of the Person object with randomly generated data. This object used for one test cycle with all serializers, then it is regenerated for the next cycle.
The measured time is for the combined serialization and deserialization operations of the same object. When serializer called the first time, it runs the longest time. This longest time span is also important and it is measured. It is the Max time. If we need only single serialization/deserialization, this is the most significant value for us. If we repeat serialization / deserialization many times, the most significant values for us are Average time and Min time.
For the Average 100% all measured times are used.
For the Average 90% the 5% slowest and 5 % fastest results ignored.
For the Average 80% the 10% slowest and 10 % fastest results ignored.
If we see significant difference between 80% and 90% average times, probably we need to increase the number of tests to get more stable and correct results.
I also provide two result sets for different test repetitions, so we can make sure the tests show stable results.
Some serializers serialize to strings, others – just to the byte array. I used base64 format to convert byte arrays to strings. I know, it is not fair, because we mostly use a byte array after serialization, not a string. UTF-8 also could be more compact format.
Again, do not take test results too seriously. I have some numbers, but this project is not the right place to get conclusions about serializer performance. You'd rather take this code and run it on your specific data in your specific workflows.
The test results below are for the 100 and 200 repetitions.
The winner is… not the ProtoBuf but NetSerializer by Tomi Valkeinen. Jil and MsgPack also show good speed and compacted strings.
The first call to serializer initializes the serializer that is why it might take thousand times faster to the next calls.
For Microsoft Avro I did not find a fast serializable method but its serialized string size is good. It has some bug preventing it from passing serialized type to the class (see the comments in code). I am really frustrated by Avro, it cannot run fast in my extremely simple code. It cannot fit in my simple serializing interface. I would appreciate the Avro experts, to optimize my code on GitHub.
Json and Binary formats bring not too much difference in the serialized string size.
Many serializers do not work well with Json DateTime format out-of-box. Only NetSerializer and Json.Net take care of DateTime format.
The NetDataContractSerializer, BinarySerializer, and Json.Net show the smallest Max times. That means they are optimal choice for cases, when we need only single serialization / deserialization cycle.
Test prints the test results on the console. It also traces the errors, the serialized strings, and the individual test times, which can be seen in DebugView for example.
So far I've seen only a waterfall methodology in the BizTalk project development. I worked for small and big companies and everywhere I saw only the waterfall. Is there something special in the BTS project that Agile is never used with it?
So far we, BizTalk developers, have all disappointments of the waterfall development: the long stages of the project, huge and outdated documentation, disagreements between users, stakeholders and developers, the bloated code, the unsatisfactory quality of the code, scary deployments and modifications. Many such problems described here by Charles Young.
Recently our team decided to use Agile principles to address these issues. We had our victories and our defeats, but right now we feel better on our journey to the "Agile world".
The business goal is simple, we desperately need faster development. When we deploy and test new code in hours not weeks, we make a lot more iterations, we make and find and fix a lot more errors. Which is just great. Now an error is not a catastrophe, it is a small thing. That means the more reliable code. Now our applications are more reliable and we fix errors very fast.
The main reason to use Agile is because of economics. It is not only faster and more reliable, it is cheaper.
We decided to use Agile together with SOA and microservice architecture. First we thought, the BizTalk is too heavy tool set to be used with Agile. But it happens the BizTalk Server has very special set of attributes that suits microservice architecture very well out of the box. If you think about orchestrations and ports as the microservices, this part of BizTalk fits perfectly to SOA.
Three main things keep BizTalk developers from using Agile. It is the artifact dependencies, "niche, unnecessary tools", and manual deployment.
Here I am talking only about technical side of problem. The management side will be touched later.
Dependency is the other side of the code reuse. One of the main differences between BizTalk applications and generic applications lays exactly here. The generic applications are created as a set of dll-s on C#, Java or any other programming language. In many cases we prefer to simplify things with code reuse, which creates some dependency problems, but usually this is not a big problem. What about the BizTalk applications? BizTalk Server keeps strict control of the working artifacts. Reliability is the king. We cannot just replace one buggy artifact if there is a dependency on it. If hurts redeployment but, remember, reliability is the king. Anything else is not so important. For us, the BizTalk developers, the cost of dependency is really high.
Moreover, dependency is a stopper for the SOA applications. Keep services independent, and it will be easy to modify them, to add new services, or to do the service versioning.
The BizTalk Server is a big toolset. It is impossible to hire a full team of the expert BizTalk developers. Most of the team members are not specialized in BizTalk too much.
So if we keep the technology stack limited, the time to make a new developer productive is short. Any tool can be replaced by C# code and the decision was to use a bare minimum of the BizTalk tools: Schemas, Maps, Orchestrations.
Completely prohibited are BRE, BAM, ESB Toolkit.
The custom pipelines, direct binding, and xslt are limited to the very special cases.
The differentiator was a question "Is this tool 100% necessary and does it require a special skill set?".
I am not going to start another holy war. Our decisions are based only on our use cases. In your company, in your zoo the decisions would be distinct.
How to make development iterations fast? One part of development cycle is deployment. It is not a problem at all if you write your program on Python or Go. It is not so big problem if you write your app on .NET. But in BizTalk development it is The Problem. So any methods to keep deployment quick are very important.
BTDF is a necessary tool in our case but the BizTalk PowerShell Provider is also used.
Deployment is always about dependencies. Penalize the dependencies and prize no-dependencies, that is the idea.
Service size: Service encapsulates a single business function and exposes only a single interface. This rule effectively cuts a BizTalk application into a single orchestration (or couple ports) in most cases.
Shared Contracts, API: Services must communicate only through contracts. The only permitted dependences between services are the contracts. We never share maps, orchestrations, ports, pipelines between applications. We only share schemas and API-s.
Versioning: the service upgrade is published as a new service. Once published service is never changed and could be only removed, not changed.
Tests: Tests are important parts of application. Application without tests is not approved. We need only user acceptance tests. The minimum test set should cover: Successful tests and Failure tests. Performance tests and Unit tests are not mandatory. The special test data is part of design. We tried to design our application in such a way that test data can be used in production together with production data.
Automation: Automated deployment is important part of application. Application without automated deployment is not approved.
As you can see this list has some specifics for the BizTalk projects. BizTalk is oriented to XML that's why we tell an XML Schema when we mean a Contract. In BizTalk projects it is really hard to test endpoints, so we tried to be easy with testing. The deployment of the BizTalk application is the complex and long process, so we force the deployment automation.
These rules were not easy to push in development. There were many unanswered questions on the way: What size is "small" and what is "big"? What modification is considered as a new version? What test coverage is enough?
These rules are not ideal. We had discussions, we added and removed rules, we changed them. We are still in the process.
A single service implemented by one developer.
A single service implemented in one week sprint.
Service developer is DevOp, he/she is responsible for deployment and all operations of the service in all environments, including production.
"One week" rule is just happened after couple months of experiments. We tried 2-3 days and 2 weeks. One week works because we want to keep all projects in one team. All developers work remotely, we never met in team together. And our team is not big, so "one week" was because of those factors.
A Senior developer approves the service design. This senior developer performs tests and deployment of the service in production and sign service to production.
If service developer or service senior developer is not accessible, a second developer should take responsibility for this role.
The "Senior Developer" rule is tricky. With this rule a senior developer cannot just sign documents, approve something and voila. No way. This rule effectively forces senior developer to make good code review and monitor all development steps. Officially this rule covers two short tasks, but they are short only if this team of two invests good amount of time to communicate all details of the service.
The true heart of the SOA and microservice lays in restricted dependencies. With BizTalk applications the dependency problem is even more important than in the standalone applications. The BizTalk controls dependencies and prohibits many shortcuts permitted in the standalone applications. So eliminating dependencies simplifies development and makes possible to make simple services, the microservices, which is our goal.
Any changes in the shared resources, any changes that could go outside of the service boundaries must be approved by the whole team. All team members should agree on this change. Any team member can veto the change.
For example, we implemented the shared infrastructure for logging. The first approach was to create the shared library (dll) and force everybody to use it. It was vetoed. The second approach was to create a special logging services and expose it as a single endpoint. It was also vetoed, because the code to use this service was too big. The next try was to use log4net or NLog and standardize the log format only. Then there was the next attempt and more. Now we discuss the InfluxDB, but what is matter, now we know much-much more what we need in reality and what we don't need.
These rules are not ideal. For example, we still struggling with the Wrong Requirements problem. How to fix this problem?
Our rule set is too long. Now we consider to join the Test and Deployment rules.
This rule set works for our team. What is special about our team? A half of the team members are the full-time employee, which enables the DevOp and Senior Developer rules. Not sure these rules would work if all members are contractors. Our team has a big list of projects to develop and the big list of applications to support. If you have mostly applications to support, our approach is possibly not the best fit for you.
I did not describe the communications with our customers and stakeholders, which is important part of the whole picture.
I did not describe the Agile practices we use (but we definitely use the Kanban board), it is not the point of this topic.
We were happy with our management, which took a risk of changes in processes and the team. Now the management is happy (hmm… almost happy) with BizTalk development and operations. Now we are SUPERFAST team! We are not sharks, not yet, but we are not jellyfishes, not anymore.
Integration and Microservices by Charles Young.
I have got an interesting request to compare and choose the right integration technology for one of my customer.
By the end of the day the technologies were limited by NServiceBus and MassTransit systems only.
The comparing process is simple. We create a sieve and filter the technologies through it. In the end we have got several, one or even zero winner.
The filters are the mix of technology, life cycle or old man questions. The whole filter process looks unscientific and unsystematic but it is sane (I hope) and simple.
What systems do we integrate?
What are your development resources: the team skills, the team agility, the team size?
What is the life horizon of the integration? 1, 2, 5, 10 years?
The integration technology could fail just on one filter and it will be enough to filter it out of competition.
So how NServiceBus and MassTransit compete there? They are very similar in technology aspects and a non-technology factor should give us the winner. It is the life horizon factor.
Do I need to comment on those graphs? Probably not.
There is a company that supports the NServiceBus, it is the Particular Software. The MassTransit is supported by open source community only, that had shown a lot of enthusiasm at early years but pretty much stopped by now. The key developers of the MassTransit have moved on some other projects.
I did not work with any of this system and, by any means, I do not choose the "best" integration system. Some of the features of one system could be much better then other, it doesn't matter now. I do not choose the better system, I do chose the failed system only for my specific requirements.
It happens the customer wants the integration code works and constantly improves at last next 5 years. From this perspective the MassTransit failed.
The domain standard schemas are the repositories of domain knowledge. The message schemas are schemas for the real data transferring between real systems. Use the domain standard schemas as a reference model for your schemas. Do not use the domain standard schemas as your message schemas.
EDI to XML or EDI to SQL?
In the EDI processing we need to transform data from the EDI format to the data formats of our applications and back.
The most popular transformation is using an intermediate XML format. We use the XML Schema and the XSLT standards to transform one XML format to another XML format. Is it the best way?
Now I'm going to step back and look at the EDI document structure a little bit.
Does the EDI structure resemble the XML structure or the SQL structure?
Does the EDI structure resemble the XML structure? It is not. The XML relations are expressed by nesting structures. For example in XML the Order Detail records are nested inside an Order record. In EDI the Order Detail and Order segments are not nested, the relations are defined the same way as it is in SQL by some correlated IDs in related segments.
So, EDI structure resembles the SQL structure not the XML structure.
Also remember our final goal, which is the application data in the SQL format. Can we just bypass the XML format and transform EDI directly to SQL format?
This is more natural. We throw out two complex EDI to XML and XML to SQL transformations and replace them by single EDI to SQL transformation. Why the EDI to XML and XML to SQL are so complex? Because we have to map the reference relations to the nesting relations or vise versa. It is not simple. There are the whole books teaching us tips and tricks for these transformations.
Why EDI to SQL transformation is simple? Because it is one to one mapping, a segment to a record and an element to a field, the SQL-like EDI structures to the SQL application structures.
The is one problem with this new approach. We have to create a code for the EDI to SQL transformation. It is not a hard problem if we use the contemporary techniques like Linq or Entity Frameworks. Those techniques look competitive even against the adapters implemented in specialized integration systems like the BizTalk Server of Mule ESB. Code is pretty straightforward if we use two step transformation.
The first step is to transform EDI segments and elements to the SQL records and fields that mirror of the EDI structure (EDI SQL), and the second step is to map those SQL records to the SQL records of our application database (App SQL).
This sample shows us how we could simplify our solutions, i.e. how to do some real architecture.
Is there something special in the BizTalk Server application deployment? Why is it so special?
For the .NET applications the live is simple. There is an exe file and maybe several additional dll-s. Copy them and this is pretty much all deployment we need.
The BizTalk Server requires the dll-s placed in GAC and registered in the special Management database. Why this is required? Mostly because the BizTalk automatically hosts applications in the cluster and because of the reliability. BizTalk application maybe is not easy in deployment but it also not easy to break. For example, if the application A is related to the application B, BizTalk will prevent us from removing B accidentally.
This Why? question is a big theme and I will not cover it here.
Another factor is the BizTalk has many pieces that require a special treatment in deployment and run-time.
Yet another factor is the BizTalk application integrates the independent applications / systems, that means the application should take care of many configuration parameters, as endpoint addresses, user names and passwords and so on.
As a result the BizTalk application deployment is complex, deployment is error prone, slow, unreliable, requires a good amount of resources.
And look here, it is a savior, the BizTalk Deployment Framework.
The BizTalk Deployment Framework (BTDF) is an essential tool in arsenal of the BizTalk Server developers and administrators. It solves many problems, it speeds up the development and deployment enormously.
It is a definitely a main face in the BizTalk Server Hall of Fame.
BTDF was created by Scott Colestock and Thomas Abraham.
It is an open source project despite the fact, that BTDF is more powerful, reliable and thorough than the most of commercial BizTalk Server third-party tools. I think it is fair to donate several dollars to those incredible guys on the Codeplex. Just think abut days and months we save in our projects, in our private life.
Giulio Van for contributing ideas and bug fixes.
And how exactly BTDF saves our life?
BTDF is installed and tuned up. Now it is time to deploy an application.
The deployment was successful and I've got the long output, what exactly was done in this deployment.
Here is a log. I've marked my comments in the deployment log with "+++". The full redeployment lasted 2 minutes.
If I perform the same amount of tasks manually, I would spent at last 3-4 additional minutes fully concentrated on this long list of deployment tasks. The probability of missing some step would be quite high as the probability of errors. With BTDF I start deployment and free to do another tasks.
So my gain is 3+ minutes on each deployment. Risk of an error is zero, everything is automated.
There is one more psychological problem with manual deployment. It is complicated and it requires the full concentration. As a developer I am concentrated on the application logic and a manual deployment task breaks my concentration. Each time after deployment I have to concentrate on my application logic again. And here is where the development performance goes down to the hell.
Restarts all host instances, only those are used in this application; restart IIS, if I deploy web-services, all with the "Bounce BizTalk" command.
Terminate all orchestration and messaging instances remained after tests.
Install the .NET component assemble into GAC.
Include the modified configuration parameters into a binding file.
Update SSO with modified configuration parameters.
All this with one click.
You do not have to use the slow BizTalk Administrative Console to do those things anymore.
Check my comments in the BTDF deployment log. Several automated tasks are like blessing! We do not create the drop folders anymore, do not assign permissions to those folders. BTDF makes this automatically.
We do not care about undeployment and deployment order, BTDF makes everything right.
We do not stop and start ports, orchestrations, applications, host instances, IIS. Everything is automated.
Remember, I told you the BizTalk application deployment is a little bit complicated? All those application components in this list require a little bit different treatment in deployment. In a simple application we do not use all those components, a typical application uses maybe 1/3 or 1/2 of them, but you have an idea.
How to tune up BTDF?
One thing I love in BTDF is the Help. It is the exemplary, ideal, flawless help.
If you never tried BTDF, there is a detailed description of all parts, all tasks. Moreover there is the most unusual part, the discussions of the BTDF principles and the BizTalk deployment processes. I got there more knowledge about the BizTalk Server deployment than in the Microsoft BizTalk Server Help - official documentation.
BTDF Help helps if you are a new user or if you use BTDF several years in row. Descriptions are clear, they are not stupid and arranged in clear hierarchy. BTDF Help is one of the best. You are never lost.
Of course there is a detailed tutorial in Help and the sample applications.
OK Now we have to start with tuning.
Forget about managing different binding files for each environment. Everything is inside one Excel table. BTDF will pass parameters from this table into binding file and other configuration stores.
Excel helps when we compound parameters from several sources. For example we keep all file port folders under one root. The folder structure below this root is the same in all environments, only the root itself is different. So there is a parameter "RootFolder" and we use it as a part of the full folder paths for all file port folders. For example, we have a "GLD_Samples_BTDFApp_Port1_File_Path" parameter which defined in the cell with formula: = "RootFolder" + ""GLD_Samples_Folder" + "BTDFApp_Folder" + "Order\\*.xml" (of course in the Excel cell it would be something like = C22 & C34 & C345 & "Order\\*.xml"). If we modify the RootFolder path, all related folder paths will be modified automatically.
Settings for different environments: In our Excel table we copy-past a column of one of the existed environment to a new column for a new environment. Then we modify the values in this new column. And again, this is an Excel table, and if we define single RootFolder variable for a new environment and voila, all file port paths for this environment are modified.
Now we have to pass those configuration parameters to the binding files, right?
Now we save this binding file with the PortBindingsMaster.xml name.
That is pretty much everything we need. Now execute the Deploy BizTalk Application command and application is deployed.
Deployment into a Production environment is different. It is limited in the installed tools and we have to do additional installation steps. BTDF creates an msi and command file. This msi includes all additional pieces we need to install. We don't have manually add resources to the BizTalk application in the Administrative Console anymore.
Conclusion: BizTalk Deployment Framework is a mandatory tool in the BizTalk development. If you are a BizTalk developer you must use it, you must know it.
Here I am going to show details of the BizTalk Server Publishers and Subscribers. We will investigate all of them step by step.
There are two main publishers: receive locations and orchestrations. The real number of publishers is bigger but let's talk about it later.
Note the PassThruReceive pipeline does not promote any additional properties.
Receive Port properties: ReceivePortID, ReceivePortName.
We can use these properties for subscriptions.
We'll see several ID properties, like ReceivePortID, which are used internally for the binding subscriptions. Not sure it is a good idea to use them explicitly.
Let's send a message to this shape and check the properties promoted on this port shape.
There are two promoted properties: Operation and SPTransportID. The SPTransportID is the ID of this send shape in orchestration, which is a constant in the current deployment.
Note inconsistency: the ReceivePortName is promoted for the receive port but an orchestration send shape name is not promoted, neither an orchestration name. So we can set up the filter expression for the receive port for the port name, but cannot set up it for the orchestration name. This promotion makes sense as a part of the binding design, but this inconsistency is worth to mention.
There are three main subscribers: send ports, send port groups, and orchestrations.
Note: If we unenlist a send port, its subscription is removed from a subscription list.
Any messages addressed to its ID will be routed properly. Technically it means this property should be promoted in a published message and this message will be routed to the port.
If we unenlist the send port group or all of its send ports, its subscription is removed from the subscription list.
Send port group behaves as a single send port. it creates a separate subscription. The messages will be delivered to the ports included in this group for the group subscription. Also messages will be delivered to the individual ports for the port subscriptions. If you have the same filters on the group and on a send port included in this group, the messages will be delivered twice.
Rule of thumb: create filter on the individual send ports OR on the send port group, and never on both.
If the orchestration is unenlisted, its subscription is still here under Disabled status, it is not removed as it happens for the send port.
If the receive shape of the orchestration is untyped and has the Specify later binding, the subscription includes the ReceivePortID. It is an ID of the orchestration receive shape.
If the receive shape of the orchestration is typed and has the Specify later binding, the subscription includes the ReceivePortID and the MessageType.
Note: the AND predicate is used.
If the receive shape of the orchestration is typed and has the Direct binding, the subscription includes the MessageType. The ReceivePortID in not in subscription anymore.
Note: The send port and the send port group get the default subscriptions on IDs, which we cannot remove. These subscriptions is to the SPTransportID and to the SPGroupID respectively. The orchestration has the default subscription to the ReceivePortID but not for the Direct binding.
for defining the domain specific standards.
Let's talk about the domain standards. EDI, RosettaNet, NIEM, ebXML, Global Justice XML Data Model, SWIFT, OpenTrave, Maritime Data Standards, HIPAA, HL7, etc. If we look at those standards, we see that schemas embrace the domain knowledge in form which can be formally and officially validated. [In this article I discussed those standards in more details: Domain Standards and Integration Architecture] The XML Schemas are very helpful for such tasks.
Compare standards which defined in form of the XML schemas and in form of the documents. It is almost impossible to verify if the data satisfy the standard or not if we use the text document where this standard is defined. And it is possible to validate it and validate it automatically, if we use the XML Schemas.
The domain specialists use XML Schemas to define standards in unambiguous form, in machine verifiable format.
Those schemas tend to be large, huge and very detailed. And it is for very good reasons.
But if we start to use XML Schemas for the first task, for processing XML documents in our programs, we need something different, we need the small schemas. In the system integration we need small schemas.
I want to emphasize it. If you work with a hundred values and use a schema with a thousand nodes, it is completely wrong. It smells all around, it intoxicates all your code. You don't want to know how programmers call this type of code.
We don't need an abundance of HIPAA schemas in most applications. We only need a small portion of schema to validate or transform the significant for this application part of schema.
We upload the megabyte size schemas, we perform mapping for these huge schemas, and it lasts for eternity and it consumes huge amount of CPU and memory.
For the most integration projects we don't want to validate data to satisfy the standard. We want to transfer data between systems as fast as possible with minimal development effort.
How to work with those wealthy schemas? How to do our integration fast at run-time and in development?
First we have to decide, does our application require the whole schemas or not?
If the answer is "No" we could read further to solution.
Solution is to simplify the schema. Cut out all unused parts of schema.
The first step in our simplification is to decide which parts of original schema we want to transfer further, want to map to another schema. We keep these parts unchanged and we simplify all other unnecessary schema parts.
The second step is to research if the target integrated system perform the data validation of the input data or not. Good system usually validates input data. Validation includes the data format validation (is this field integer, date type or does it match a regex expression?), the data range (is this string too long or is this integer too big?), the right encoding (is this value belong to the code table?), etc.
If the target system performs this validation, it doesn't make sense to us perform the same data validation on the integration layer. We just pass the data without any validation to the target system. Let this system validate data and decide what to do with errors: send errors back to the source system or try to repair or something else. Actually it is not good architecture, if an intermediary (our integration system) is trying to do such validations and decisions. It means spreading the business logic between systems where target system delegates the data validation logic to intermediary. The integration system deals with data validation only if it needed.
Now let's be more technical. The next example is implemented in the BizTalk Server and the HIPAA schemas, but you can use the same principles with other systems and standards.
The first step in the schema simplification is the structural modification. It is pretty simple. We replace the unused schema parts with <any> tags [http://www.w3.org/TR/xmlschema-0/#any]. If we are still want to map this schema part but without any details, we can use the Mass Copy functoid.
The second part of the schema simplification is the type simplification.
Open schema again with Schema Editor, make any small change and undo it. Editor will recalculate type information and pops up the Clean Up Global Data Types window. Check all types and click OK.
This cleans up all unused Global Data Types.
Previously we replaced all those types with "xs:string" type and those types are not used anymore.
It takes 5 minutes for this replacement. What is the result?
The modified schema is twice smaller.
- is the dll size with original schema.
- is dll size with modified schema.
The assembly for modified schema also cut twice in size.
Result is not bad for 5 min job.
How these simplified schemas change our performance?
All projects with schemas and maps are compiled in Visual Studio notably faster. I like this improvement as a developer.
How about the run-time performance?
I have made a simple proof of concept project to check the performance changes.
The project compounded of two BizTalk applications and two BizTalk Visual Studio projects. Do not do this in production projects! One Visual Studio solution should keep one and exactly one BizTalk application.
Each project keeps one HIPAA schema, one very simple schema, one "complex" map (HIPAA schema to HIPAA schema), and one simple map (HIPAA schema to the very simple schema).
The first project works with original HIPAA schema and the second project with simplified HIPAA schema.
Build and Deploy one project.
After deployment import the binding file which you find in the project folder. Create the In and Out folders and apply necessary permissions to those folders. Change the folder paths in the file locations for your folders.
There is also a UnitTests project with several unit tests. Change folder paths in the test code.
Then delete the application and deploy second BizTalk project and perform tests again.
Do not deploy both projects side by side.
Note: Before each test start the Backup BizTalk job to clean up a little bit the MessageBox.
Tests for 1, 10 and 100 messages did not show visible difference. The difference could be noticeable in my environment in 1000 message and 3K message batch tests. The above table shows the test result for 3K batch tests.
The performance gain is about 10%. It is not breathtaking but anyway it is not so bad for the 5 minutes effort.
Conclusion: The schema type simplification is worth to do if the application expects the sustainable high payloads, the high peak payloads, and everywhere you want to get the best possible performance.
Sometimes we need to make complex decoding in the data transformations. It could happen especially in the big EDI documents as HIPAA.
In one example we need to decode a field. We have the source codes and the target codes for this field. The number of both codes is small and the mapping is one to one or many to one (1-1, M-1). One of the simplest solution is to create a Decode .NET component. Store the code table as a dictionary and decoding will be fast. We could hard code the code table, if codes are stable, or extract it in cache, reading it from a database or a configuration file.
The next example is on the opposite side of complexity. Here we need to decode several fields. Target codes related to several source codes/values (M-1). It is not the value-to-value decoding but this decoding also includes several if-then-else conditions, which opens a can of worms with 1-M and M-M cardinality.
Moreover the code tables are big and cannot be placed in the memory.
We can implement this decoding with numerous calls to the database to get the target codes and perform these calls inside a map or inside a .NET component. As a result for each document transformation we calls database many times.
But there is another method of implementing this without flooding the database with these calls. I call this method the "SQL Decoding".
Load the source message to SQL database.
Execute encoding as a SQL operation or series of operations.
Extract the target message back from the SQL.
We can do all structure transformations in maps (XSLT) and perform only decoding in SQL form. Or we also can do some structure transformations in SQL. It is up to us.
It is fast and highly optimized.
It does not flood database with numerous calls.
It nicely utilizes the SQL engine for complex decoding logic.
Steps 1 and 3 can be not simple.
In real life we usually don't have a clear separation between our scenarios and the intermediate solutions can be handy. As an example we can load and extract not a whole message but only part of it, related to decoding.
Personally, I use the SQL Decoding in the most complex cases where mapping takes more than 2 days of development.
If you are familiar with LINQ, you can avoid steps 1 and 3 and execute set operations directly on the XML document. I personally prefer to use LINQ. But if XML document is really big, SQL approach works better.
Conclusion: If we need a complex decoding/encoding of the Xml document, consider to use the set operations with SQL or LINQ.