text:
Letter From Gympie 2021
BOOKS OLD AND RARE
Gone to Gympie.
Reflective bookseller Barry Long moved to Gympie Queensland late in the year 2015. The following year he wrote to some friends and relatives, describing his discoveries and experiences. A year later, he extended the correspondence to include customers. A monthly letter from Gympie went out to customers, friends and relatives of the reflective bookseller.
January 2017 Number One.
Inventory is the elephant in an online bookseller's room. On arrival at Gympie, a human assembly line unloaded box after box of books from two furniture vans. Not only was the relocation of so many books an unforgettable experience. It was a time when the full meaning of the word inventory emerged looming larger than ever before. Calling for cataloguing, pricing, description, curating, imaging, cleaning and stocktaking, the inventory elephant may have saddled me with a job for life.
Thoughts of a lifetime looking after an inventory would be daunting without incentives to keep going. One incentive is when new information, an exciting discovery, arises. Another is when an old and rare book becomes especially relevant in today's world. And now there is the prospect of a newsletter to write. But when a customer experiences my own enjoyment reading the same book, bookselling seems the best job in the world.
FEBRUARY 2017 Number Two
Cane toads were as still as statues and silent as sentinels. One evening shortly after arriving in Gympie, my first cane toad appeared. Following what I was told was the required procedure, I removed the creature. An unexpected sight on returning then confronted me. About a dozen stiffly upright cane toads were spaced out, poised motionless to attention on the lawn. Glaring eyes cast disapproval of what had befallen their friend and relative.
Months later, when winter came, after an overgrown lawn was mown, they appeared again, gathered outside the back door. Despite having lost their grass cover, they no longer seemed reproachful. Going towards the door, I saw them shaking. One youngster, hunched at the sliding door jamb, was shivering all over. Sensitive to cold, they were seeking the warmth of the house.
March 2017 Number Three
Sundays shine by clearing away the rust of the week. (1) But Saturday mornings sparkle with a sense of freedom as their prospect of something different emerges. The Saturday Book's editor, John Hadfield, claimed there was nothing like the Saturday Book. He was right, and it often seems there is nothing like Saturday itself.
The Saturday Book was founded in 1941, the second year of the second world war. Its editors attributed the book's success to the same highest common factor – the pictorial partnership of Olive Cook and Edwin Smith – " an art form all of their own" of pictorial anthologies. Each Saturday Book ranges from works of art, articles, essays and fiction to social commentary, oddities and curiosities. John Hadfield identified three points to be made about the book. First, there is nothing like it. Second it defies description. Third, it was distinguished at the time by an arguably longer life than any other annual miscellany.
Hadfield called it the Saturday Book after humming a song he heard in a London music hall. The song's title, Sweet Saturday Nights , had " the right air of week-end relaxation."(2)
(1) Lamb, Charles. The Essays of Elia : The Superannuated Man (1833) (2) Russell, Leonard, editor. The Saturday Book ( 1941 - 1951 ). Hadfield, John, editor.The Saturday Book ( 1952 - 1975 ). The Best of the Saturday Book ( 1981 ).
April 2017 Number Four
More birds crowd into a Bunya pine than you might think possible. My house is perched on filling tipped into a former rock quarry hillside. The house may be commonplace, but the view is not. A back deck overlooks Bruce Highway traffic winding and undulating towards Gympie's Normanby Bridge. Colourful hinterland hilltops form a peaceful backdrop, as does a large Bunya pine tree nearby. Alongside a railway viaduct over the notoriously flood prone Deep Creek, the Bunya pine proudly challenges the floodway. Accommodating seemingly unlimited bird life, the tree's somewhat bare branches are distinctive. How so many birds disappear into it at night and manage to squeeze in is a mystery.
May 2017 Number Five
The ceremony for Gympie's Anzac Day was at the Memorial Gates. Memorial Lane opposite Gympie's Memorial Park has ironwork gates, sandstone pillars, coloured murals and inscribed plaques. It commemorates local people in service from the Boer War to Afghanistan. Situated alongside the central business district, Memorial Park is embellished by a nineteen twenties band rotunda. Terracotta tiles, a gabled roof, skirting and balustrades have created a structure of classic style. The skirting fretwork features musical instruments.The park's appearance is enriched by simplicity of design and seasonal plant changes. Its green buffalo grass defying parching heat, the park offers calmness , especially when afternoon shadows are cast.
June 2017 Number Six
A gold rush boom town, Gympie was not ordinary but orderly. Named after a stinging tree, Gympie by most accounts accommodated its share of frontier hardships well. An article by nineteenth century travel writer Gilbert Parker reads: " …Gympie is one of the prettiest places in Australia. When I was there the many hills were green, and quiet habitations dotted their sides…..At night the main street is paraded by the miners, the most orderly set of fellows that one could find…" Despite candidly also describing Gympie's vices, trickery and behaviour of its people, Parker wrote : "Gympie tries to be right. ….The place outwardly is one of the most orderly to be found…". (1)
(1) Parker, Gilbert. Round the Compass in Australia ( not dated but c.1892 ).
Plan to fulfill your gap year carefully (if you can get one). Those fortunate enough to achieve a gap year are often compelled to overcrowd it. Henry David Thoreau's gap year communing with nature from a log cabin lasted two years and two months from 1845 to 1847. Borrowing an axe to build the cabin, he carried most of the materials on his back. Thoreau's counsel to those who do not keep pace with their companions has endured ever since. Recognising we might be hearing a different drummer, he suggested stepping to the music heard, no matter how far away it sounds. Observing lives frittered away by detail, Thoreau loved living to what he referred to so brilliantly as a broad margin. Why, he asked, should we live with such hurry and waste of life? (1)
(1) Thoreau, Henry David. Walden (1980).
An author envisaged a great boomerang watering Australia's north. His biographer hoped Ion L Idriess would be acknowledged as the man who had the vision to dream of a way to water the desert. (1) Idriess lectured for ten years on his idea before his book was published. He proposed turning the rivers of Australia's north back into a system of tunnels. (2) Articles in the Sun newspaper also featured the plan during the year 1936. A somewhat similar official plan, the Bradfield Scheme, has been mooted for over seven decades.
(1) Eley, Beverley. Idriess (1995) (2) Idriess, Ion. The Great Boomerang (1941).
July 2017 Number Seven
The more you know about your area, the more you become part of it. (1) Local histories remain relevant, no matter when they were written. Arguments at the town hall and rezoning disputes are only some of the ingredients. Athletes who represented their district, state or nation in their sport, communities in times of distress and disaster, so many other things of unusual interest in a particular locality, and the service of people to their neighbours and community all contribute to the mix. Reading about those who once lived in particular houses and streets makes a daily walk or jog knowledgeable. Enlivening readers' understanding of their communities, local histories invite you to keep returning to them to dip inside their content again and again. Kept prominently on the shelf, a local history makes a statement of commitment to a place. It can also become a topic of discussion.
(1) Holthouse, Hector. Gympie Gold (1973)
August 2017 Number Eight
This was the thirty sixth year of the Gympie Muster, a three day Australian music festival. The atmosphere was enjoyable as you threaded your way through the street party celebrating the three day event.
Australia's best architect designed Gympie's Town Hall. Gympie's Town Hall was designed by Australian colonial architect John James Clark, renowned for the Melbourne Treasury, Brisbane Treasury and many other fine public buildings in Australia and New Zealand. Andrew Dodd, Clark's biographer cites Florence Taylor, Australia's first qualified female architect, describing Clark as "… the greatest Australian practitioner in the glorious profession of architecture." in the journal Building 1917. Built in 1890, the building is distinguished by its clock tower and iron clad belfry. University of Melbourne Archives have the original drawing of the building as it was proposed. (1)
(1) Dodd, Andrew. J J Clark Architect of the Australian Renaissance (2012).
Listen for the past and present sounds of Gympie. I am standing near the front steps of Gympie Post Office opposite the Town Hall. Within no more than a few minutes walking distance are two monuments to James Nash. One is alongside Memorial Park, once the gully where Nash first found gold. The second is opposite the Post Office and in front of the Town Hall. It is not far from where Nash worked his claim so successfully. October sixteen 1867, the day James Nash's discovery of gold was announced, became Gympie's birthday. Imagine the sounds of this hilly area 150 years ago: a virtual army of swagged tent dwelling prospectors arriving to set up camp, the sound resonating of thousands of diggers working, shovels scraping, picks chipping and winches creaking.
By contrast, little imagination is needed to appreciate the Town Hall clock chimes. Here in the central business district, the chimes are at their distinctive best. Their clarity of sound dignifies your presence by reminding you where you are. Sometimes the chimes seem to smarten me up by sharpening my sense of place and widening awareness of my surroundings.The chimes are comforting, especially late at night in summer. When everything is sublimely quiet, their sound can be heard through an open window. It adds a dimension to a place renowned for its visually appealing setting.
September 2017 Number Nine
Read not only for knowledge and stimulation. The potential of books to influence individuals is infinite. As a senior school student, Arthur Conan Doyle was strongly influenced by Macaulay's Literary and Historical Essays (1850). Arriving in London at the age of sixteen, the first thing he did after housing his luggage was to visit Macaulay's grave in Westminster Abbey. Doyle described Macaulay's grave as " the one great object of interest" London held for him. "And so it might well be when I think of all I owe him." he wrote many years later. What did Doyle,the creator of Sherlock Holmes, owe Thomas Macaulay? Knowledge and stimulation of fresh interests, character and personal qualities radiating from Macaulay's writing all raised Doyle's debt of gratitude.(1)
(1) Doyle, Arthur Conan.Through the Magic Door (1907).
October fifteen 2017 number Ten
Gympie purple has burst forth. Walking to the shopping centre is refreshing and the temperature comfortable as the sun starts to set. The first week in October brought with it swathes of Jacaranda trees flowering purple. October and November will be followed in December and January by flaming red Poincianas. Gympie has just celebrated its 150th anniversary. Done and dusted, the 2017 Gympie Muster has come and gone, the haystacks, gear and street furniture stored away . In its place was the Gold Rush festival celebrating Gympie's 150 years of age. A crowd thronged Mary Street to enjoy the procession and later watch Jimmy Barnes and other performers.
Milkshakes on Saturdays seemed something special. Saturday mornings - to a ten year old – were accompanied by the allure of milkshakes. Following the morning's tennis coaching, milkshakes at Calleo's in Sydney's Charing Cross precinct were a reward for achievement. Shining in its décor, the shop was at the intersection of Bronte Road, Carrington Road and Victoria Street. Socialising occurred here as get- togethers of youthful spirits squeezed and jostled into the milk-bar's booths. Plans were made for the afternoon as the prospect of movies loomed.
Cinema would be followed by rallies of Bondi Junction junior street gangs in the Coronet's adjoining laneway. Re-enactments of The Iron Claw starring Victor Jory and other movie serials would be staged in nearby Queen's Park Waverley.
October thirtieth 2017 Number 11
The illicit sale of chicken sandwiches was one of my first Saturday jobs. Reporting early one Saturday morning to a house in an inner suburb, I once sought early work placement in sales. A man at the door wearing trousers supported by braces over his singlet asked me my age. He waved me through after I nervously mumbled something indistinct in reply. The house was crammed full of people making chicken sandwiches, wrapping them in parcels and packing them into carrying baskets.
A van was waiting and we were soon crowded into it with our baskets and offloaded into city hotels. Some hotels providing counter lunches - I quickly learnt - disallowed selling sandwiches. Throughout the day I was warned off and dodged threats by hotel managers and publicans. But most of the sandwiches sold. By late afternoon after settlement I had money in my pocket.
November 2017 Number 12
Rising over Skyring Creek, the newly extended M1's elevation and sweep are far reaching. Imprinted into Gympie's history, the Skyring family's pioneering settlement and leadership was exemplified by empathy with the aboriginal community. Impressed into rolling hills and open forest, the surrounding hinterland scenery is distinguished by greenness of vegetation.
A paperback writer's words were crafted inside a garrett. During my final primary school vacation I worked with a school friend for his dad. We delivered ice to houses without fridges. Using grappling hooks we carried blocks of ice from his dad's delivery truck to put inside household ice chests. The fourth storey of one terrace house was a roof top attic bed- sitter. To reach the entrance to this low ceiling studio I climbed to the top of a steep narrow staircase. Once inside, I had a bird's eye view of Sydney harbour through a bay window. Nearby someone was typing away at a table. After pushing the ice into a chest, I peered excitedly over the typist's shoulder. While the rattling keys clattered onwards, I asked what he was writing. "Westerns", he replied tersely.
Longing to linger, I wanted desperately to watch a wild west story being created. But the delivery truck horn was beeping and my friend's dad could not be kept waiting. Afterwards I browsed all the paperback westerns for sale on a newsagents' shelves. Admiring their illustrated covers, I wondered whether they were written in the garrett.
For a very long time in history, people usually read aloud.No one seems to know precisely when things changed. But a recorded instance of silent reading is apparent in 6 : 3 of Confessions by Augustine of Hippo. Calling to visit Bishop Ambrose in the fourth century AD, Augustine is somewhat intrigued to find him reading silently. He speculates why Ambrose would be doing such a thing.
December 2017 Number 13
A few weeks ago I met a storyteller by the Noosa river at Tewantin. I was relaxing on the river bank when a stranger sat down next to me and started talking. As a tour operator on nearbyFraser Island, he told aboriginal legends to tourists. Shortly afterwards we were joined by a young couple from the Netherlands who were on his camping expedition. Under a clear blue sky we listened as he narrated myths and legends of animals and birds and sun and moon. Although he was not aboriginal, the wealth of imagination in aboriginal legends is something he seemed to have captured.
You should have your head read. My mother would sometimes tell my father he should have his head read. Mum seemed to be quite angry with dad whenever she admonished him with this advice. Too young to understand what mum was talking about, I was old enough to be puzzled. My imagination stretched to visualise people peering over the top of dad's head. But how could someone read it? When mum said they looked for bumps and felt them, I became even more confused. Mum's remedy for her perception of dad's failings was once a common saying. It evolved from phrenology. Atlantic magazine ( January 2014) has a comprehensive article on the history and substance or otherwise of the science.
What you read recently is more interesting than your small talk. "I might run out of conversation", someone said to me, "I'll be at a loss to come up with something worthwhile to say."
"Didn't you just finish reading a biography?" I asked. " yes, but it was nothing special " came the reply
"But you read it only recently, so it beats your small talk. One of my favourite authors would bet it does", I said. (1)
(1) Doyle, Arthur Conan, Through the Magic Door (1907).
©Barry Long 2017
Read more Letters Here
PO Box 1047, Gympie, Qld 4570, Australia | 07 5482 1214


text:
Jadla is a village in Shaheed Bhagat Singh Nagar district of Punjab State, India. It is located  away from postal head office Lassara,  from Nawanshahr,  from district headquarter Shaheed Bhagat Singh Nagar and  from state capital Chandigarh. The village is administrated by Sarpanch an elected representative of the village.

Demography
As of 2011, Jadla has a total number of 895 houses and population of 4442 of which 2309 include are males while 2133 are females according to the report published by Census India in 2011. The literacy rate of Jadla is 80.39%, higher than the state average of 75.84%. The population of children under the age of 6 years is 460 which is 10.36% of total population of Jadla, and child sex ratio is approximately 847 as compared to Punjab state average of 846.

Most of the people are from Schedule Caste which constitutes 47.01% of total population in Jadla. The town does not have any Schedule Tribe population so far.

As per the report published by Census India in 2011, 1447 people were engaged in work activities out of the total population of Jadla which includes 1293 males and 154 females. According to census survey report 2011, 81.69% workers describe their work as main work and 18.31% workers are involved in Marginal activity providing livelihood for less than 6 months.

Education
The village has a Punjabi medium, co-ed upper primary with secondary/higher secondary school founded in 1929. The school provide mid-day meal as per Indian Midday Meal Scheme and also provide free education to children between the ages of 6 and 14 as per Right of Children to Free and Compulsory Education Act.

KC Engineering College and Doaba Khalsa Trust Group Of Institutions are the nearest colleges. Industrial Training Institute for women (ITI Nawanshahr) is  and Lovely Professional University is  away from the village.

Landmarks
Gurudwara Sahib Shiri Guru Ravidass Ji Maharaj, Gurudwara Singh Sabha, Gurudawa Shahib Shaheed Singh, Shiv Mandir and Mata Parvati Mandir, Panj Peer Darwar, are religious sites. The village has 3 banks and 4 ATM facility and Police Post. Police Post Jadla is located near Mini Bus stand Palace Road Jadla. Police Post Jadla is  under The jurisdiction of Police Station Sadar Nawanshahr.

Transport
Nawanshahr railway station is the nearest train station however, Garhshankar Junction railway station is  away from the village. Sahnewal Airport is the nearest domestic airport which located  away in Ludhiana and the nearest international airport is located in Chandigarh also Sri Guru Ram Dass Jee International Airport is the second nearest airport which is  away in Amritsar.

See also
List of villages in India

References

External links
 Tourism of Punjab
 Census of Punjab
 Locality Based PINCode

Villages in Shaheed Bhagat Singh Nagar district

redpajama_set_name: RedPajamaWikipedia


text:
Afghanistan's fall after US withdrawal rattles nerves in Iraq
Taliban's lightning sweep into major Afghan cities and the government's swift fall took the world by surprise
Taliban fighters patrol in Kandahar, Afghanistan, on Tuesday. The movement's co-founder Abdul Ghani Baradar, declared victory on Monday and an end to the decades-long war in Afghanistan. EPA
Sinan Mahmoud
Follow the latest updates on Afghanistan here
Iraqis are watching the Taliban's takeover of Afghanistan with dismay amid concern their country could face a similar fate if the US withdraws its troops this winter.
The group's lightning sweep into Kabul and the Afghan government's fall took the world by surprise.
It occurred two weeks before US forces had been due to complete their withdrawal from the country, and some prominent Iraqis say the hasty departure has eroded trust in Washington.
"After all that happened in Afghanistan, anyone among us still considers the US training and consultancy camps [in Iraq] as a gain?" said Izzat Al Shabandar, an Iraqi Shiite politician.
"Of course, some will say the situation in Iraq is different [from the one in Afghanistan], but we see that America is America and that its goals are not different."
In 2003, the US led an international coalition to invade Iraq and topple Saddam Hussein's regime, claiming it was developing and stockpiling weapons of mass destruction. None were ever found in Iraq and the country was plunged into chaos after the invasion.
After nearly nine years, the US withdrew from Iraq, leaving behind a small number of troops to protect the US Embassy and to train and assist Iraqis.
Mesopotamian treasures on display at the Iraqi Museum in Baghdad. The National Museum, known as the Cradle of Civilisation Museum, reopened at the end of February 2009 after its ancient treasures were looted in the aftermath of the US-led invasion in 2003.
But in late 2014, combat troops returned to fight ISIS, which then controlled about a third of the country, as the US-trained Iraqi security forces melted away.
Despite the defeat of ISIS by the end of 2017, about 5,000 troops remained, along with others from the international coalition to suppress the terrorist group.
Since then, Iran-linked Shiite politicians and militias have called for US troops to leave.
Tension has risen since Iran's top military commander, Qassem Suleimani, was assassinated soon after arriving in Baghdad. He was killed in a US drone strike alongside influential Iraqi militia leader Abu Mahdi Al Muhandis near the city's airport.
Militias have since staged rocket and bomb attacks against American troops, the US embassy and convoys supplying international coalition bases.
Last year, former US President Donald Trump decided to reduce the number of soldiers in Iraq to 2,500. In July, both countries agreed to end the US combat mission in December, shifting the role to an advisory and educational one.
Chaos at Kabul airport as desperate Afghans try to escape the Taliban
Afghanistan latest: Taliban offer 'general amnesty' as US negotiates over Kabul airport
Explained: The Taliban takeover and what's next for Afghanistan
Some have sounded warnings a withdrawal of US troops could lead to the resurgence of ISIS or a scenario akin to the current situation in Afghanistan.
The writer Tariq Jawhar Sarmami, who lives in Erbil, said the developments "must not go unnoticed or to be seen as isolated or small event, especially by the leaders in the Kurdistan region and Iraq generally".
"So, it is not inconceivable that the United States follow the same approach with Iraq next year," Mr Sarmami said. "Whenever it finds a suitable alternative it will abandon its partners."
But Kifah Mahmoud, an independent political researcher, disagreed.
"The situation in Iraq is different," he said. "Iraq is a strategic project for America that is associated with the New East Project and spreading democracy in the region."
"I believe that the circumstances that led to 2014's events do not exist now and Iraqi security forces, Popular Mobilisation Forces [an umbrella group composed largely of Iran-backed Shiite militias], and Kurdish peshmerga are more capable and experienced."
Washington's failure to prevent the Taliban's resurgence has fuelled anti-American conspiracy theories by Iran-backed Shiite militias in Iraq. They have accused the US of orchestrating the rise of ISIS.
"What's happening in Afghanistan reminds us of what happened here in 2014," said Adham Al Bahadly, who runs one of the many PMF news platforms on social media.
"But the difference is that our martyrs shed their blood to save Iraq." He was referring to the PMF, which was established in 2014 to fight ISIS.
"Without that blood we would have seen the same pictures in Iraq."
Some pro-Shiite militias platforms shared the images of desperate Afghans clinging to a departing US military plane. They sounded a warning to the pro-US camp in Iraq that it faced the same fate.
"Americans evacuated their dogs from Afghanistan but not Afghan contractors and those co-operated with them, [American] agents [in Iraq] take note," one post says.
Updated: August 18th 2021, 1:12 PM
IraqAfghanistanTaliban
Exclusive: Abu Dhabi blast survivors recall moment deadly Houthi strike hit
Suspect in murder of Egyptian speedball champion has confessed, say police
As snow hits Lebanon, people burn rubbish and clothes to stay warm

redpajama_set_name: RedPajamaCommonCrawl


A £75,000 road improvement scheme for B2 Drumnacanvy Road, Portadown will start next Tuesday 20 March 2018.
The Department for Infrastructure said the scheme will 'improve forward visibility and horizontal alignment including localised road widening in the vicinity of Knocknamuckly Lane.
The road will be closed from 20 March for approximately six weeks.
Signed diversions will be in place via Knock Road and Bleary Road for traffic travelling from Gilford Road towards Drumnacanvy and via Long Lane for traffic travelling towards Gilford.
The Department has programmed the works operations and traffic management arrangements to minimise any inconvenience, however, drivers should expect some delays and are advised to leave additional time when travelling in the area. The dates and duration of the works will be dependent on the progress of the scheme and on favourable weather conditions.


text:
Missouri 4-H Dairy Judging competes at national contest
Published: Wednesday, Nov. 27, 2019
MADISON, Wis. – At the recent National 4-H Dairy Cattle Judging Contest, the Missouri 4-H team placed fourth in Ayrshires, seventh in Brown Swiss and eighth in oral reasons. The team finished 12th overall.
"The team didn't have its best day judging but finished with some outstanding individual honors," said Karla Deaver, team coach and University of Missouri Extension 4-H youth development specialist in Lawrence County.
"While the honors and awards are wonderful, the true outcomes from judging team participation come from developing life skills in confidence, public speaking, decision-making and problem-solving," Deaver said. "Traveling and competing as a team exposes youth to new opportunities, introduces them to other youth and adults who share their interest in judging, and to additional judging opportunities during their collegiate careers."
In the indivdiual standings, Bailey Groves, daughter of Todd and Sheila Groves of Billings, was first in Ayrshires, second in Brown Swiss and third in oral reasons, averaging a 45.8 points out of 50 on five sets. "This is the highest individual in oral reasons Missouri 4-H has had since 1997, and possibly ever," said Deaver.
Hala Edquist, daughter of Eric and Cheryl Edquist of Mountain Grove, came in second in Ayrshires and 10th in Brown Swiss. Rounding out the Missouri team were Blake Wright, son of Larry and Jodi Wright of Verona, and Nicolas Dotson, son of Earl and Nicole Dotson of Marionville.
Contestants placed one cow and one heifer class in Ayrshires, Brown Swiss, Guernseys, Holsteins and Jerseys and gave five sets of reasons. The contest took place at the 2019 World Dairy Expo, Sept. 29-Oct. 5 in Madison, Wisconsin.
The team is coached by Deaver and MU Extension dairy specialist Ted Probert, Wright County.
"As coaches, Ted and I have been proud of how these young adults have learned from each other and grown as we have trained for competition," Deaver said. "We are excited to watch these young people as they continue to grow and develop skills, and we are proud to call them Missouri 4-H Dairy Judging alumni."
The Missouri 4-H Dairy Judging Team receives support from FCS Financial, the Missouri Holstein Association, Missouri Dairy Association, Bill Darr and the Missouri 4-H Foundation, with additional support from Friends of Missouri 4-H Dairy Judging. The team is currently seeking additional sponsors. For more information about the dairy judging program, contact Ted Probert, MU Extension in Wright County, at 417-349-4134, or Karla Deaver, MU Extension in Lawrence County, at 417-466-3102.
Photo available for this release:
https://extensiondata.missouri.edu/NewsAdmin/Photos/2019/4H/2019DairyJudging.jpg
The Missouri 4-H Dairy Judging Team includes, from left, Ted Probert, coach; Blake Wright; Hala Edquist; Bailey Groves; Nicolas Dotson; and Karla Deaver, coach.
Karla Deaver
Missouri 4-H
Use arrows left and right to move between the items.


text:
\section{Introduction}
Video captioning, the task aims to automatically generate natural language descriptions for videos, has received increasing attention in computer vision and machine learning. Even though our community achieves the significant advance in visual recognition~\cite{he2016deep,ren2015faster} and natural language understanding~\cite{bahdanau2014neural}, video captioning is still a very challenging task and far away from satisfactory for it not only requires a thorough understanding of the input videos, but also requires step-by-step visual reasoning along the generation process.

\input{figures/fig1.tex}

As illustrated in Figure~\ref{fig:intro}, to describe the video, we human may have the following reasoning process: 1) identifying the subject to describe of the video, \textit{i.e.}, ``man'', 2) inferring what is the man doing, \textit{i.e.}, ``shooting'', 3) identifying what is the object of ``shooting'', \textit{i.e.}, ``basketball'', 4) generating the final description ``a man is shooting a basketball'' by inserting several function words. In a nutshell, there are three fundamental reasoning mechanisms involved: 1) Locate one region to generate visual words, 2) Relate pairwise regions to generate action words, 3) Generate function words according to the language context.

Most existing video captioning methods~\cite{venugopalan2015sequence,donahue2015long} follow the encoder-decoder framework, where a CNN is employed as an encoder to produce the video features and an RNN is employed as a decoder to generate the captions. Those methods usually neglect the nature of the above human-level reasoning, thus hurting the explainability of the generation process.
Even though there are some recent works have explored the visual reasoning in visual question answering~\cite{andreas2016neural,hu2017learning,9009518} and visual grounding~\cite{cirik2018using,liu2019learning,hong2019learning} by decomposing the questions or referring expressions into a linear or tree reasoning structure with several neural modules, the situation in the video captioning is more challenging because 1) unlike still images, videos contain richer visual content thus requiring more complex visual reasoning over both space and time, 2) unlike questions or referring expressions which are given in advance, the video descriptions are not available during the inference. Therefore, the model must dynamically compose the reasoning structure along the generation process.

To tackle the above two challenges, we propose a novel video captioning framework named Reasoning Module Networks (RMN).
Firstly, to perform visual reasoning over both space and time, our RMN employs three fundamental spatio-temporal reasoning modules: a)~\textsc{Locate} module to locate one single region over the video by a spatial-temporal attention, thus generating the visual words, \textit{e.g.}, ``man'' and ``basketball'' in Figure~\ref{fig:intro};
b)~\textsc{Relate} module to relate pairwise regions over the video by first detecting the object of each frame, and then modeling the action by pairing two frames, thus generating the action words, \textit{e.g.}, ``shooting'' in Figure~\ref{fig:intro}; and
c)~\textsc{Func} module to generate the function words according to the language context, \textit{e.g.}, ``a'' and ``is'' in Figure~\ref{fig:intro}.
Secondly, to compose the reasoning structure along the generation process, our RMN employs a dynamic and discrete module selector.
To jointly train both the three modules and the selector in an end-to-end manner, we adopt the recently proposed Gumbel approximation~\cite{jang2016categorical} to make the discrete sampling process differentiable, and then constraint the module selector with a linguistic loss of part-of-speech (POS) tag labels.

We validate the effectiveness of the proposed RMN by conducting extensive experiments on two widely-used datasets MSVD~\cite{chen2011collecting} and MSR-VTT~\cite{xu2016msr}. RMN outperforms the state-of-the-art video captioning methods on most metrics.
Qualitative results indicate the generation process is explicit and explainable.

Our main contributions are three-fold:
1) We propose a novel framework named reasoning module networks (RMN) for video captioning with three \textit{spatio-temporal} visual reasoning modules;
2) We adopt a \textit{discrete} module selector to \textit{dynamically} compose the reasoning process with modules;
3) Our RMN achieves new state-of-the-art performance with an explicit and explainable generation process.

\section{Related Work}
\subsection{Video Captioning}
There are two main directions to solve the video captioning problem. In the early stage, template-based methods~\cite{kojima2002natural,guadarrama2013youtube2text}, which first define a sentence template with grammar rules and then aligned subject, verb and object of the sentence template with video content, were widely studied. Those methods are hard to generate flexible language due to the fixed syntactic structure of the predefined template.
Benefit from the rapid development of deep neural networks, the sequence learning methods~\cite{venugopalan2015sequence,yao2015describing,pan2017video} are widely used to describe the video with flexible natural language, most of these methods are based on the encoder-decoder framework.
\cite{venugopalan2015sequence} proposed S2VT model which regards the video captioning task as a machine translation task. ~\cite{yao2015describing} introduced a temporal attention mechanism to assign weights to the features of each frame and then fused them based on the attention weights. \cite{li2017mam,chen2019motion} further applied spatial attention mechanisms on each frame.

Recently, \cite{wang2019controllable} and \cite{hou2019joint} proposed to leverage Part-of-Speech (POS) tags to boost video captioning. \cite{wang2019controllable} encodes the predicted POS sequences into hidden features, which further guides the generation process. \cite{hou2019joint} mixes word probabilities of multiple components at each timestep conditioned on the inferred POS tags. However, both of them lack the reasoning capability for rich video content. On the contrary, we propose three well-designed reasoning module networks that correspond to three fundamental reasoning mechanisms.

\subsection{Neural Module Networks}
Neural module networks is a general framework that explicitly models the compositionality of languages by decomposing the network into neural modules. It has been widely used in visual question answering and visual grounding.
\cite{andreas2016neural} employs an off-the-shelf parser to parse the questions into a tree, leading to brittleness caused by parsing errors.
\cite{hu2017learning} trains an RNN to decode the language into the sequence, thus requiring extra human annotations.
\cite{hu2018explainable} removes language parser and additional annotations by using a soft and continuous module layout.
However, the case of video captioning is more complex since there are no fully observed captions during inference thus all the above methods are not applicable. To this end, we design a dynamic module selector to construct the reasoning procedure step-by-step during the generation process.

Recently, the pioneering works~\cite{liu2018context,8684270,yang2019learning,tian2019image} are trying to adopt neural module networks into image captioning.
However, their modules are designed to produce several types of features and their module compositions relied on a soft attention mechanism.
As a contrast, our RMN employs several sophisticated spatio-temporal reasoning modules to perform more complex visual reasoning over videos, and designs a discrete and dynamic module selector to make a selection at each step, making the generation process explicit and explainable.

\section{Approach}

\input{figures/fig2.tex}
In this section, we will describe the proposed Reasoning Module Networks (RMN) in more detail.
Figure~\ref{fig:networks} gives a walk-through example of our RMN at timestep $t$. Our RMN can be divided into four stages: Encoding (Section~\ref{sec:3.1}), Module Reasoning (Section~\ref{sec:3.2}), Module Selection (Section~\ref{sec:3.3}), and Decoding (Section~\ref{sec:3.4}). In Section~\ref{sec:3.5}, we detail the joint training strategy for both reasoning modules and the module selector.

\subsection{Encoder}
\label{sec:3.1}
For the given video of $N$ frames, we first represent it with three types of features, \textit{i.e.}, appearance features $\bm{V_a}$ extracted from a 2D-CNN, object features $\bm{V_o}$ extracted from a R-CNN on each frame, and motion features $\bm{V_m}$ extracted from a 3D-CNN. Note that to model temporal information for those visual features, we have post-processed $\bm{V_a}$ and $\bm{V_m}$ with Bi-LSTMs. And $\bm{V_o}$ contains an additional dimension on space.

For the previous generated words $\{w_1, \cdots, w_{t-1}\}$, we encode them by a LSTM (denoted as enLSTM) which takes the global visual feature $\bar{\bm{v}}$, the last generated word embedding vector $\bm{e}_{t-1}$, and the last timestep hidden state $\bm{h}^{de}_{t-1}$ of deLSTM (cf. Eqn.~\eqref{eq:delstm}) as input:
\begin{equation}
    \bm{h}^{en}_t = \textbf{enLSTM}(\bar{\bm{v}}, \bm{e}_{t-1}, \bm{h}^{de}_{t-1}).
\label{eq:enlstm}
\end{equation}
It is worth noticing that since $\bm{h}^{en}_t$ encodes rich information from history, it will be used to guide the reasoning process and the module selection.

\subsection{Reasoning Modules}
\label{sec:3.2}
As mentioned in the Introduction, there are three fundamental reasoning mechanisms involved in video captioning.
Therefore, we design three corresponding reasoning modules. In the following, we will first introduce the attention functions used in the modules, and then describe each module in detail.

\subsubsection{Attention Functions}
We follow the widely used additive attention formulation~\cite{bahdanau2014neural}:
\begin{equation}
    \textbf{A}(\bm{V}, \bm{q}) = \softmax(\bm{w}^T_1 \tanh (\bm{W}_2 \bm{V} + \bm{W}_3 \mathds{1}^T \bm{q})) \bm{V},
\end{equation}
where $\bm{V}$ and $\bm{q}$ are values and queries of attention, $\bm{w}_1$, $\bm{W}_2$, and $\bm{W}_3$ are trainable parameters, $\mathds{1}$ is an all-one vector.
Note that, the attention can be executed on any dimensions, therefore, we further define $\textbf{AoS}(\cdot)$ as the Attention over the dimension on Space, and define $\textbf{AoT}(\cdot)$ as the Attention over the dimension on Time.

\subsubsection{\textsc{Locate} Module}
It is designed to generate visual words, \textit{e.g.}, ``man'' and ``basketball''. Generating this type of words requires the model to attend on one specific region over both space and time. Therefore, we first apply an Attention over Space (AoS) for object features $\bm{V_o}$ and then apply an Attention over Time (AoT) together with $\bm{V_a}$, formally:
\begin{equation}
    \bm{v^l_t} = \textbf{AoT}(\textbf{AoS}(\bm{V_o}, \bm{h}^{en}_t) \oplus \bm{V_a}, \bm{h}^{en}_t),
\end{equation}
where $\oplus$ denotes concatenate operation.

\subsubsection{\textsc{Relate} Module}
It is designed to generate action words, \textit{e.g.}, ``riding'' and ``shooting''. Modeling action requires reasoning over both space and time.
Take Figure~\ref{fig:intro} as an example, to generate the word ``shooting'', we must be aware that the man is holding a basketball, and then notice that the basketball is flying to the basket, finally we can inference that the man is ``shooting''.
It means we must connect two scenes at different time.
To achieve this reasoning mechanism, we insert a pairwise interaction (\textit{i.e.}, concatenate every possible pair of two tensors) between the AoS and AoT, formally:
\begin{equation}
\begin{split}
    \bm{v^r_t} & = \textbf{AoT}(\textbf{P}(\bm{M}, \bm{M}), \bm{h}^{en}_t), \\
    \bm{M} & = \textbf{AoS}(\bm{V_o}, \bm{h}^{en}_t) \oplus \bm{V_m},
\end{split}
\end{equation}
where $\textbf{P}(\cdot, \cdot)$ denotes the pairwise interaction function that $\textbf{P}_{ij}(\bm{A}, \bm{B})=\bm{A}_i \oplus \bm{B}_j$.

\subsubsection{\textsc{Func} Module}
It is designed to generate function words to complete the whole sentences. Since the function words only require language information, thus we propose to recall the history decoder cell states of deLSTM (cf. Eqn.~\eqref{eq:delstm}) to generate the current words by an AoT, formally:
\begin{equation}
\begin{split}
    \bm{v^f_t} & = \textbf{AoT}(\bm{C}, \bm{h}^{en}_t), \\
    \bm{C} & = [\bm{c}^{de}_1, \cdots, \bm{c}^{de}_{t-1}].
\end{split}
\label{eq:func}
\end{equation}

So far, we have detailed the three proposed reasoning modules. Next, we will describe how to make a selection of those modules at each timestep.

\subsection{Module Selector}
\label{sec:3.3}
As illustrated in Figure~\ref{fig:networks}, the module selection consists of two steps. First, we calculate a score for each module to measure the probability of each module could be selected.
Second, we sampling one determined module based on those scores with Gumbel Softmax~\cite{jang2016categorical} strategy.

In detail, we formulate the scoring function $\textbf{S}(\cdot, \cdot)$ as:
\begin{equation}
    \textbf{S}(\bm{h}, \bm{v}) = \fc(\tanh(\fc(\bm{h}) + \fc(\bm{v}))).
\end{equation}
Thus we can get $s^l_t=\textbf{S}(\bm{h}^{en}_t, \bm{v}^l_t)$, $s^r_t=\textbf{S}(\bm{h}^{en}_t, \bm{v}^r_t)$, and $s^f_t=\textbf{S}(\bm{h}^{en}_t, \bm{v}^f_t)$ for the three \textsc{Locate}, \textsc{Relate}, and \textsc{Func} modules, respectively.

After that, the straightforward selection method is directly choosing the module with the maximum score.
However, it will block the gradients off the module selector for the function $\arg\max$ is non-differentiable.
To this end, we apply a relaxation between forward pass and backward pass, formally:
\begin{equation}
\begin{split}
    & \mathrm{Forward}: \bm{z}_t = \arg\max(\log([s^l_t, s^r_t, s^f_t]) + G), \\
    & \mathrm{Backward}: \tilde{\bm{z}}_t = \softmax((\log([s^l_t, s^r_t, s^f_t]) + G)/\tau),
\end{split}
\end{equation}
where we take an approximation that relaxes the discrete one-hot decision vector $\bm{z}_t$ into a continuous $\tilde{\bm{z}}_t$.
$G$ is the Gumbel noise drawn from i.i.d. Gumbel(0, 1). $\tau$ is a temperature parameter to control the strength of softmax.
Please refer to \cite{jang2016categorical} for more mathematical details.

With this decision vector $\bm{z}_t$, we can get the final visual reasoning result as:
\begin{equation}
    \bm{v}_t = \bm{z}_t \otimes [\bm{v}^l_t, \bm{v}^r_t, \bm{v}^f_t],
\label{eq:vt}
\end{equation}
where $\otimes$ denotes inner product. It is worth noticing that $\bm{z}_t$ is an one-hot vector, thus only one determined module will be selected in the forward pass.

\subsection{Decoder}
\label{sec:3.4}
For each timestep $t$, we decode the final visual reasoning result by a LSTM (denotes as deLSTM):
\begin{equation}
    \bm{h}_t^{de}, \bm{c}_t^{de} = \textbf{deLSTM}(\bm{v}_t, \bm{h}_t^{en}).
\label{eq:delstm}
\end{equation}
Recall that the hidden and cell states were used in Eqn.~\eqref{eq:enlstm} and Eqn.~\eqref{eq:func}.
Further, we decode the word probability $\bm{p}_t$ as:
\begin{equation}
    \bm{p}_t = \softmax(\textbf{MLP}(\bm{v}_t \oplus \bm{h}_t^{en} \oplus \bm{h}_t^{de})),
\end{equation}
where $\textbf{MLP}$ is two-layers with $\tanh$ as activation function.

\subsection{End-to-End Training}
\label{sec:3.5}
Thanks to the Gumbel-Softmax strategy, our RMN model can be trained in an end-to-end manner.
Given a video and the corresponding ground-truth caption $\{w^*_t\}$ for $t \in [1, T]$, where $T$ is the sentence length, the objective is to minimize the cross-entropy loss function given by:
\begin{equation}
    \mathcal{L}_{cap} = - \sum _{t=1}^{T}\log\bm{P}_t(w^*_t).
\end{equation}

Further, to ensure the module selector sticking on the linguistic structure, we propose to supervise the module selection by extra Part-of-Speech (POS) tag labels. Accordingly, we apply a Kullback-Leibler Divergence (KLD) loss to make the predicted module decision vector $\tilde{\bm{z}}$ and the ground-truth tag distribution $S^*=\{s^*_t\},\, t\in[1, T]$ as close as possible, formally:
\begin{equation}
    \mathcal{L}_{pos} = - \sum_{t=1}^{T} \mathrm{KLD}(\tilde{\bm{z}}_t \| \mathtt{one\_hot}(s^*_t)).
\end{equation}

To collect the ground-truth $S^*$, we first detected POS tag labels by Spacy Tagging Tool\footnote{https://spacy.io}, and then assigned [NN*] and [JJ*] to the \textsc{Locate} module, [VB*] to the \textsc{Relate} module, and the other to the \textsc{Func} module.

Therefore, the overall loss function of our RMN model is given by:
\begin{equation}
    \mathcal{L} = \mathcal{L}_{cap} + \lambda \mathcal{L}_{pos},
\end{equation}
where $\lambda$ is a trade-off weight.
\section{Experiments}

\input{tables/ablation.tex}

\subsection{Datasets and Metrics}
We conduct experiments on two widely used video captioning datasets with several standard evaluation metrics to verify the effectiveness of our proposed method.

\subsubsection{Datasets}
\noindent{\textbf{MSVD}.} The MSVD dataset \cite{chen2011collecting} consists of 1,970 short video clips selected from Youtube, where each one depicts a single activity in the open domain, and each video clip is annotated with multi-lingual captions. Since we only consider the English captions in this paper, each video clip has roughly 41 descriptions. To be consistent with previous works, we split the dataset to 3 subsets, 1,200 clips for training, 100 clips for validation, and the remaining 670 clips for testing.

\noindent{\textbf{MSR-VTT}.} The MSR-VTT \cite{xu2016msr} is a large-scale dataset for the open domain video captioning, which consists of 10,000 video clips from 20 categories, and each video clip is annotated with 20 English sentences by Amazon Mechanical Turks. There are about 29,000 unique words in all captions. Following the existing works, we use the standard splits, namely 6,513 clips for training, 497 clips for validation, and 2,990 clips for testing.

\subsubsection{Evaluation Metrics}
We use several widely used automatic evaluation metrics to evaluate the quality of the generated captions, \textit{i.e.}, BLEU \cite{papineni2002bleu}, METEOR \cite{banerjee2005meteor}, CIDEr \cite{vedantam2015cider}, ROUGE-L \cite{lin-2004-rouge}. Most of these metrics are originally proposed for machine translation or image captioning, the higher score indicates better quality of the captions.

\subsection{Implementation Details}
\subsubsection{Dataset Preprocessing} We first convert all captions to lower case and remove punctuations, then we truncate the captions with more than 26 words and zero pad the captions with less than 26 words. The vocabulary size is set to 7,351 for MSVD and 9,732 for MSR-VTT with removing the words appear less than twice and five times respectively.

\subsubsection{Feature Extraction}
In our experiments, we use InceptionResNetV2 (IRV2) \cite{szegedy2017inception} as 2D CNN and I3D \cite{carreira2017quo} as 3D CNN to extract appearance features and motion features respectively, then we equally-spaced 26 features for each video. The IRV2 is trained on ILSVRC-2012-CLS image classification dataset \cite{Russakovsky2015ImageNetLS} and the I3D is trained on Kinetics action classification dataset \cite{1705.06950}. We adopt Faster-RCNN \cite{ren2015faster} which is trained by \cite{Anderson2017up-down} to extract 36 region features for each frame (26 equally-spaced frames for each video).

\subsubsection{Training Details}
Our model is optimized by Adam Optimizer ~\cite{kingma2014adam}, the initial learning rate is set to 1e-4. For the MSVD dataset, the hidden size of the LSTM is 512 and the learning rate is divided by 10 every 10 epochs. For the MSR-VTT dataset, the hidden size of the LSTM is 1,300 and the learning rate is divided by 3 every 5 epochs. During testing, we use beam search with size 2 for the final caption generation.

\subsection{Ablation Study}

\input{figures/pie.tex}
\input{figures/word_cloud.tex}
\input{figures/example.tex}

As shown in Table~\ref{table:ablation}, we compare our RMN against a set of other ablated models with various settings:
(1) \textbf{RMN (\textsc{Locate})}: the model that only deployed with Locate module;
(2) \textbf{RMN (\textsc{Relate})}: the model that only deployed with Related module;
(3) \textbf{RMN (S)}: the model that softly fuses three modules, \textit{i.e.}, $\bm{z}_t$ in Eqn.~\eqref{eq:vt} is computed as $\softmax(\log([s^l_t, s^r_t, s^f_t])$;
(4) \textbf{RMN (H)}: the model that hard selects one of three modules;
(5) \textbf{RMN (S+L)}: the model that softly fuses three modules and trained by linguistic loss;
(6) \textbf{RMN (H+L)}: the model that hard selects one of three modules and trained by linguistic loss.
According to the results shown in Table~\ref{table:ablation}, we have the following observations.

\subsubsection{Effect of Reasoning Modules}
RMN (\textsc{Relate}) consistently outperforms RMN (\textsc{Locate}) on both MSVD and MSR-VTT. This is because the \textsc{Relate} module models much richer visual action information than \textsc{Locate}, leading to performance improvement over RMN (\textsc{Locate}).

RMN (S) outperforms RMN (\textsc{Locate}) and RMN (\textsc{Relate}) under most metrics. This indicates that by fusing different types of reasoning mechanisms, the model can generate better video descriptions.

\subsubsection{Effect of Gumbel Strategy}
Comparing RMN (H) with RMN (\textsc{Locate}) and RMN (\textsc{Relate}), we observe opposite results on MSVD and MSR-VTT.
To figure out the reasons for this observation, we count the proportion of the three modules occupied in Figure~\ref{fig:pie}. We can find that without the linguistic loss, RMN (H) only employs two modules, \textit{i.e.}, \textsc{Locate} module and \textsc{Relate} module, indicating the failure of training.

Therefore, the reason for the opposite results is that RMN (H) failed in training on MSVD while succeeded on MSR-VTT.
Actually, even though Gumbel strategy can relax the discrete decision to continuous decision, it still requires large training examples.
Therefore, RMN (H) underperforms RMN (\textsc{Locate}) and RMN (\textsc{Relate}) on MSVD which only contains 1,200 training samples, while outperforming them on MSR-VTT which contains 6,513 training samples.

\input{tables/sota.tex}

Similarly, even though RMN(S) surpasses RMN(H) under some metrics, but if we employ the linguistic loss to make sure both soft fusion and hard module selector works well, RMN (H+L) outperforms RMN (S+L) with a margin.

\subsubsection{Effect of Linguistic Loss}
Comparing RMN (H+L) with RMN (H), as well as comparing RMN (S+L) with RMN (S), we find that the linguistic loss consistently improves the performance, indicating the importance of linguistic information for video captioning.

We can also observe the consistent results in Figure~\ref{fig:pie}. By imposing the linguistic loss, we can enforce the module selector sticking with the linguistic structure, and the proportion of each module is approaching to the ground-truth.

Further, we carried out the word cloud statistics as shown in Figure~\ref{fig:wordcloud}.
We can find that the three modules have a strong pattern: the \textsc{Locate} module mainly generates visual words, \textit{e.g.}, ``man'' and ``person''; the \textsc{Relate} module mainly generates action words, \textit{e.g.}, ``playing'' and ``talking''; the \textsc{Func} module mainly generates function words, \textit{e.g.}, ``about'' and ``to''. It indicates that the model has learned linguistic knowledge correctly.

\subsection{Comparison with State-of-the-Art}

We compare our proposed RMN with the most recent state-of-the-art methods on MSVD and MSR-VTT datasets. According to whether they leverage the POS labels, we categorize them into two groups: 1) traditional encoder-decoder based models MAM-RNN~\cite{li2017mam}, RecNet~\cite{wang2018reconstruction}, MARN~\cite{pei2019memory}, and OA-BTG~\cite{zhang2019object}, and 2) POS strengthened model POS-CG~\cite{wang2019controllable} and Mixture~\cite{hou2019joint}.

As shown in Table~\ref{table:sota}, we can find that: 1) the methods that leverage POS labels outperform the methods without POS information, and 2) our proposed reasoning module network outperforms the methods with POS labels on most metrics and achieves new state-of-the-art.
It is worth noting that CIDEr is proposed for captioning task specifically, and is considered more consistent with human judgment. Our model achieves the best CIDEr score on both datasets, which demonstrates our RMN model can generate captions that more in line with human descriptions.

\subsection{Qualitative Analysis}
In this section, we would like to investigate the generation process of our model by qualitative results.
Here we provide some video captioning examples in Figure~\ref{fig:example}.
As expected, our module selection is explicit and reasonable, for example, we compose ``man'', ''flute``, and ``phone'' with the \textsc{Locate} module, ``playing'', ``riding'', and ``talking'' with \textsc{Relate} module, and ``a'', ``of'', and ``the'' with \textsc{Func} module.

The above observations suggest that the generation process of the proposed RMN is totally explicit, since at each timestep the module selects one determined module to produce the current word. In addition, each module is well-designed to perform spatio-temporal visual reasoning, \textit{e.g.}, the spatio-temporal attention for \textsc{Locate} module and pairwise interaction reasoning over both space and time for \textsc{Relate} module, indicating that our model is explainable.

\section{Conclusion}
In this paper, we proposed a novel reasoning neural module networks (RMN) for video captioning that performs visual reasoning on each step along the generation process. Specifically, we designed three sophisticated reasoning modules for spatio-temporal visual reasoning. To dynamically compose the reasoning modules, we proposed a discrete module selector which is trained by a linguistic loss with a Gumbel approximation. Extensive experiments verified the effectiveness of the proposed RMN, and the qualitative results indicated the caption generation process is explicit and explainable.

\section*{Acknowledgments}
This work was supported by the National Key R\&D Program of China under Grant 2017YFB1300201, the National Natural Science Foundation of China (NSFC) under Grants U19B2038, 61620106009 and 61725203 as well as the Fundamental Research Funds for the Central Universities under Grant WK2100100030.

redpajama_set_name: RedPajamaArXiv


text:
Japan's NHK begins world's first 8K TV broadcasts
8k nhk japan
By AdvancedSetup
August 14, 2016 in General Chat
AdvancedSetup
Location: The United Federation of Planets
Japan's public broadcaster NHK has begun the world's first regular TV satellite broadcasts in 8K resolution. The "Super Hi-Vision" test channel launched on Monday and has a mix of content in both 4K and 8K resolution, which at 7680 x 4320 pixels is four times as sharp as 4K and 16 times as sharp as 1080p; the spec also allows for 22.2-channel audio.
Can't imagine trying to edit 8K video on a Computer. The 4K video format when editing in raw native mode can put a stress on even the latest high end computers. You can have 70+ Mbps in 4K mode recordings. On my 4K video camera it creates about 60MB for 8 seconds of video.
Malware Hunters
Interests: Airgun shooting
So in a few years i have to buy a new TV again ....
daledoc1
5 hours ago, pondus said:
I think there's a method to the manufacturers' motives.
David H. Lipman
Location: Jersey Shore USA
Interests: Malware Research, dSLR Photography, Numismatics & Surf Fishing
Yep. That's me.
I'm so old, deaf, dumb and blind, none of these technological innovations will be of any benefit. My planned obsolescence approaches at break-neck speed.
Heck, I'm about to toss out (finally) one of *2* old CRT TVs.
I noticed a big difference in living room space when my 32" 75 kg monster went out the door


With this period property you have the benefit of having your own front door, high ceilings and a very practical layout.
This property is well proportioned in size and does require significant modernisation. It is ideal for someone looking to really put their stamp on it and make it their own.
You also have the luxury of having direct access to a private rear garden, great to enjoy in the warm summer months.
Properties like this are popular with first time buyers, investors and also people looking to tap into the great local buy to let market. They do not stay on the market for long so enquire today to avoid disappointment.


text:
New insights into the earliest events of seed germination
By Isabel	24/01/2020 March 31st, 2020 Botany, News, Plant Science
Thiol redox switches regulate how plant seeds use their stored energy reserves
Plant seeds may strike the casual observer as unspectacular – but they have properties that are nothing short of superpowers. In a dry state they can store their energy for years and then suddenly release it for germination when environmental conditions are favourable. One striking example is the "super bloom" in the Death Valley National Park, when seeds that have endured the dry and hot desert for decades suddenly germinate at rainfall followed by a rare and spectacular desert bloom several months later. Seeds conserve a fully formed embryo, which only continues growing when conditions are right for it to do so. This may be the case only years – or in more extreme cases even centuries – later.
Seed germination is controlled by several plant hormones, which are researched intensely. However, not much was known about the processes that need to take place to allow the hormones to function. How is energy in the seed made available? How can energy metabolism be started early and efficiently? An international team of researchers has now been looking into these questions.
Using a new type of fluorescent biosensors, the researchers observed, in living seed cells, both energy metabolism and the so-called redox metabolism, which relies in sulphur. The researchers discovered that when the seeds came into contact with water, energy metabolism was established in a matter of minutes, and the plant cells' "power stations" – known as mitochondria – activated their respiration. The researchers also found out which molecular switches are activated to enable energy to be released efficiently – with the so-called thiol-redox switches playing a central role.
"By looking into the very early processes of germination control, we can gain a better understanding of the mechanisms driving seed germination," says Prof. Markus Schwarzländer from the University of Münster, who led the study. "In future we could think about how such switches could be used in crop biotechnology." The results of the study could be of relevance in farming, when seeds need to keep their germination vigour for as long as possible on the one hand, but should also germinate in synch and with minimal losses on the other hand. The study has been published in the journal PNAS.
Background and method
In order to be able to observe the activities taking place in the energy metabolism, the researchers visualized under the microscope adenosine triphosphate (ATP), the general currency for energy in the cell, and Nicotinamide adenine dinucleotide phosphate (NADPH), the electron energy, in the mitochondria. They compared seeds from thale cress: both dry seeds and seeds "imbibed" with water.
To find out whether the redox switches are important for kick-starting germination, the researchers deactivated specific proteins using genetic methods and then compared the reaction shown by the modified seeds with that of the unmodified ones. The researchers allowed the seeds to age artificially in the laboratory, and they saw that the seeds germinated much less actively if they lacked the relevant proteins.
The researchers' next step involved so-called redox proteome analysis, i.e. they examined the relevant redox proteins in their entirety with the use of biochemical methods. For this purpose, they isolated active mitochondria and flash-froze them in order to be able to study this state directly where the process was taking place. The researchers then used mass spectrometry methods to identify several so-called cysteine-peptides which are important for resource efficiency in energy metabolism.
"The process could be likened to the traffic control system of a large city. Before the rush hour – i.e. germination – starts, which puts large quantities of metabolites 'on the road', the traffic light and routing systems need to be switched on in the morning; and here this is done by the thiol redox switches," explains lead author Dr. Thomas Nietzel, who carried out most of the experiments as part of his PhD at the Institute of Crop Science and Resource Conservation at the University of Bonn and later as a postdoctoral researcher at the Institute of Biology and Biotechnology of Plants at the University of Münster.
Institutions involved
In addition to the University of Münster, other universities involved in the study were those of Greifswald, Bonn, Heidelberg, Aarhus (Denmark) and Lorraine (France) – as well as the Research Institute of Horticulture and Seeds in Beaucouzé (France) and the Max Planck Institute of Plant Breeding Research in Cologne.
Read the paper: PNAS
Article source: University of Münster
Image credit: Bettina Richter
germplasmplant physiologyplant scienceseed store
Previous PostOne size may not suit all
Next PostECRi communication and dissemination activity
ForestryNews
Droughts, viruses and road networks: Trends that will impact our forests
NewsPlant Science
Plants can be larks or night owls just like us
AgricultureNewsPlant Science
Green Revolution Saved Over 100 Million Infant Lives in Developing World, Yet Could Go Further


text:
//== llvm/Support/LowLevelTypeImpl.h --------------------------- -*- C++ -*-==//
//
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//===----------------------------------------------------------------------===//
//
/// Implement a low-level type suitable for MachineInstr level instruction
/// selection.
///
/// For a type attached to a MachineInstr, we only care about 2 details: total
/// size and the number of vector lanes (if any). Accordingly, there are 4
/// possible valid type-kinds:
///
///    * `sN` for scalars and aggregates
///    * `<N x sM>` for vectors, which must have at least 2 elements.
///    * `pN` for pointers
///
/// Other information required for correct selection is expected to be carried
/// by the opcode, or non-type flags. For example the distinction between G_ADD
/// and G_FADD for int/float or fast-math flags.
//
//===----------------------------------------------------------------------===//

#ifndef LLVM_SUPPORT_LOWLEVELTYPEIMPL_H
#define LLVM_SUPPORT_LOWLEVELTYPEIMPL_H

#include "llvm/ADT/DenseMapInfo.h"
#include "llvm/Support/MachineValueType.h"
#include <cassert>

namespace llvm {

class DataLayout;
class Type;
class raw_ostream;

class LLT {
public:
  /// Get a low-level scalar or aggregate "bag of bits".
  static LLT scalar(unsigned SizeInBits) {
    assert(SizeInBits > 0 && "invalid scalar size");
    return LLT{/*isPointer=*/false, /*isVector=*/false, /*NumElements=*/0,
               SizeInBits, /*AddressSpace=*/0};
  }

  /// Get a low-level pointer in the given address space.
  static LLT pointer(unsigned AddressSpace, unsigned SizeInBits) {
    assert(SizeInBits > 0 && "invalid pointer size");
    return LLT{/*isPointer=*/true, /*isVector=*/false, /*NumElements=*/0,
               SizeInBits, AddressSpace};
  }

  /// Get a low-level vector of some number of elements and element width.
  /// \p NumElements must be at least 2.
  static LLT vector(uint16_t NumElements, unsigned ScalarSizeInBits) {
    assert(NumElements > 1 && "invalid number of vector elements");
    assert(ScalarSizeInBits > 0 && "invalid vector element size");
    return LLT{/*isPointer=*/false, /*isVector=*/true, NumElements,
               ScalarSizeInBits, /*AddressSpace=*/0};
  }

  /// Get a low-level vector of some number of elements and element type.
  static LLT vector(uint16_t NumElements, LLT ScalarTy) {
    assert(NumElements > 1 && "invalid number of vector elements");
    assert(!ScalarTy.isVector() && "invalid vector element type");
    return LLT{ScalarTy.isPointer(), /*isVector=*/true, NumElements,
               ScalarTy.getSizeInBits(),
               ScalarTy.isPointer() ? ScalarTy.getAddressSpace() : 0};
  }

  static LLT scalarOrVector(uint16_t NumElements, LLT ScalarTy) {
    return NumElements == 1 ? ScalarTy : LLT::vector(NumElements, ScalarTy);
  }

  static LLT scalarOrVector(uint16_t NumElements, unsigned ScalarSize) {
    return scalarOrVector(NumElements, LLT::scalar(ScalarSize));
  }

  explicit LLT(bool isPointer, bool isVector, uint16_t NumElements,
               unsigned SizeInBits, unsigned AddressSpace) {
    init(isPointer, isVector, NumElements, SizeInBits, AddressSpace);
  }
  explicit LLT() : IsPointer(false), IsVector(false), RawData(0) {}

  explicit LLT(MVT VT);

  bool isValid() const { return RawData != 0; }

  bool isScalar() const { return isValid() && !IsPointer && !IsVector; }

  bool isPointer() const { return isValid() && IsPointer && !IsVector; }

  bool isVector() const { return isValid() && IsVector; }

  /// Returns the number of elements in a vector LLT. Must only be called on
  /// vector types.
  uint16_t getNumElements() const {
    assert(IsVector && "cannot get number of elements on scalar/aggregate");
    if (!IsPointer)
      return getFieldValue(VectorElementsFieldInfo);
    else
      return getFieldValue(PointerVectorElementsFieldInfo);
  }

  /// Returns the total size of the type. Must only be called on sized types.
  unsigned getSizeInBits() const {
    if (isPointer() || isScalar())
      return getScalarSizeInBits();
    return getScalarSizeInBits() * getNumElements();
  }

  /// Returns the total size of the type in bytes, i.e. number of whole bytes
  /// needed to represent the size in bits. Must only be called on sized types.
  unsigned getSizeInBytes() const {
    return (getSizeInBits() + 7) / 8;
  }

  LLT getScalarType() const {
    return isVector() ? getElementType() : *this;
  }

  /// If this type is a vector, return a vector with the same number of elements
  /// but the new element type. Otherwise, return the new element type.
  LLT changeElementType(LLT NewEltTy) const {
    return isVector() ? LLT::vector(getNumElements(), NewEltTy) : NewEltTy;
  }

  /// If this type is a vector, return a vector with the same number of elements
  /// but the new element size. Otherwise, return the new element type. Invalid
  /// for pointer types. For pointer types, use changeElementType.
  LLT changeElementSize(unsigned NewEltSize) const {
    assert(!getScalarType().isPointer() &&
           "invalid to directly change element size for pointers");
    return isVector() ? LLT::vector(getNumElements(), NewEltSize)
                      : LLT::scalar(NewEltSize);
  }

  unsigned getScalarSizeInBits() const {
    assert(RawData != 0 && "Invalid Type");
    if (!IsVector) {
      if (!IsPointer)
        return getFieldValue(ScalarSizeFieldInfo);
      else
        return getFieldValue(PointerSizeFieldInfo);
    } else {
      if (!IsPointer)
        return getFieldValue(VectorSizeFieldInfo);
      else
        return getFieldValue(PointerVectorSizeFieldInfo);
    }
  }

  unsigned getAddressSpace() const {
    assert(RawData != 0 && "Invalid Type");
    assert(IsPointer && "cannot get address space of non-pointer type");
    if (!IsVector)
      return getFieldValue(PointerAddressSpaceFieldInfo);
    else
      return getFieldValue(PointerVectorAddressSpaceFieldInfo);
  }

  /// Returns the vector's element type. Only valid for vector types.
  LLT getElementType() const {
    assert(isVector() && "cannot get element type of scalar/aggregate");
    if (IsPointer)
      return pointer(getAddressSpace(), getScalarSizeInBits());
    else
      return scalar(getScalarSizeInBits());
  }

  void print(raw_ostream &OS) const;

  bool operator==(const LLT &RHS) const {
    return IsPointer == RHS.IsPointer && IsVector == RHS.IsVector &&
           RHS.RawData == RawData;
  }

  bool operator!=(const LLT &RHS) const { return !(*this == RHS); }

  friend struct DenseMapInfo<LLT>;
  friend class GISelInstProfileBuilder;

private:
  /// LLT is packed into 64 bits as follows:
  /// isPointer : 1
  /// isVector  : 1
  /// with 62 bits remaining for Kind-specific data, packed in bitfields
  /// as described below. As there isn't a simple portable way to pack bits
  /// into bitfields, here the different fields in the packed structure is
  /// described in static const *Field variables. Each of these variables
  /// is a 2-element array, with the first element describing the bitfield size
  /// and the second element describing the bitfield offset.
  typedef int BitFieldInfo[2];
  ///
  /// This is how the bitfields are packed per Kind:
  /// * Invalid:
  ///   gets encoded as RawData == 0, as that is an invalid encoding, since for
  ///   valid encodings, SizeInBits/SizeOfElement must be larger than 0.
  /// * Non-pointer scalar (isPointer == 0 && isVector == 0):
  ///   SizeInBits: 32;
  static const constexpr BitFieldInfo ScalarSizeFieldInfo{32, 0};
  /// * Pointer (isPointer == 1 && isVector == 0):
  ///   SizeInBits: 16;
  ///   AddressSpace: 24;
  static const constexpr BitFieldInfo PointerSizeFieldInfo{16, 0};
  static const constexpr BitFieldInfo PointerAddressSpaceFieldInfo{
      24, PointerSizeFieldInfo[0] + PointerSizeFieldInfo[1]};
  /// * Vector-of-non-pointer (isPointer == 0 && isVector == 1):
  ///   NumElements: 16;
  ///   SizeOfElement: 32;
  static const constexpr BitFieldInfo VectorElementsFieldInfo{16, 0};
  static const constexpr BitFieldInfo VectorSizeFieldInfo{
      32, VectorElementsFieldInfo[0] + VectorElementsFieldInfo[1]};
  /// * Vector-of-pointer (isPointer == 1 && isVector == 1):
  ///   NumElements: 16;
  ///   SizeOfElement: 16;
  ///   AddressSpace: 24;
  static const constexpr BitFieldInfo PointerVectorElementsFieldInfo{16, 0};
  static const constexpr BitFieldInfo PointerVectorSizeFieldInfo{
      16,
      PointerVectorElementsFieldInfo[1] + PointerVectorElementsFieldInfo[0]};
  static const constexpr BitFieldInfo PointerVectorAddressSpaceFieldInfo{
      24, PointerVectorSizeFieldInfo[1] + PointerVectorSizeFieldInfo[0]};

  uint64_t IsPointer : 1;
  uint64_t IsVector : 1;
  uint64_t RawData : 62;

  static uint64_t getMask(const BitFieldInfo FieldInfo) {
    const int FieldSizeInBits = FieldInfo[0];
    return (((uint64_t)1) << FieldSizeInBits) - 1;
  }
  static uint64_t maskAndShift(uint64_t Val, uint64_t Mask, uint8_t Shift) {
    assert(Val <= Mask && "Value too large for field");
    return (Val & Mask) << Shift;
  }
  static uint64_t maskAndShift(uint64_t Val, const BitFieldInfo FieldInfo) {
    return maskAndShift(Val, getMask(FieldInfo), FieldInfo[1]);
  }
  uint64_t getFieldValue(const BitFieldInfo FieldInfo) const {
    return getMask(FieldInfo) & (RawData >> FieldInfo[1]);
  }

  void init(bool IsPointer, bool IsVector, uint16_t NumElements,
            unsigned SizeInBits, unsigned AddressSpace) {
    this->IsPointer = IsPointer;
    this->IsVector = IsVector;
    if (!IsVector) {
      if (!IsPointer)
        RawData = maskAndShift(SizeInBits, ScalarSizeFieldInfo);
      else
        RawData = maskAndShift(SizeInBits, PointerSizeFieldInfo) |
                  maskAndShift(AddressSpace, PointerAddressSpaceFieldInfo);
    } else {
      assert(NumElements > 1 && "invalid number of vector elements");
      if (!IsPointer)
        RawData = maskAndShift(NumElements, VectorElementsFieldInfo) |
                  maskAndShift(SizeInBits, VectorSizeFieldInfo);
      else
        RawData =
            maskAndShift(NumElements, PointerVectorElementsFieldInfo) |
            maskAndShift(SizeInBits, PointerVectorSizeFieldInfo) |
            maskAndShift(AddressSpace, PointerVectorAddressSpaceFieldInfo);
    }
  }

  uint64_t getUniqueRAWLLTData() const {
    return ((uint64_t)RawData) << 2 | ((uint64_t)IsPointer) << 1 |
           ((uint64_t)IsVector);
  }
};

inline raw_ostream& operator<<(raw_ostream &OS, const LLT &Ty) {
  Ty.print(OS);
  return OS;
}

template<> struct DenseMapInfo<LLT> {
  static inline LLT getEmptyKey() {
    LLT Invalid;
    Invalid.IsPointer = true;
    return Invalid;
  }
  static inline LLT getTombstoneKey() {
    LLT Invalid;
    Invalid.IsVector = true;
    return Invalid;
  }
  static inline unsigned getHashValue(const LLT &Ty) {
    uint64_t Val = Ty.getUniqueRAWLLTData();
    return DenseMapInfo<uint64_t>::getHashValue(Val);
  }
  static bool isEqual(const LLT &LHS, const LLT &RHS) {
    return LHS == RHS;
  }
};

}

#endif // LLVM_SUPPORT_LOWLEVELTYPEIMPL_H

redpajama_set_name: RedPajamaGithub