Who doesn't like meeting friends over a good Sunday brunch? You get to catch up while munching on some bites? I know I do. I would shovel these so-called "bites" down my stomach whenever there was a gathering after work, at a conference, at a birthday party, a wedding- practically everywhere without paying attention to the harm these "bites" were causing until my body started aching. Each time I consumed certain food products like pizza, pork fried rice with ribs or a cheeseburger with fries followed with soda I would go down after with an excruciating pain episode. Some of which landed me to the emergency room. Naturally my first instinct was to cure the symptoms rather than to modify my habits. Until it just dawned on me. If I didn't change my behavior I would eventually collapse.
The body is a perfect machine that seeks to regulate itself. Just like a car it starts giving you signals when it is not operating at its optimum level. It starts breaking down until it shuts down completely if you ignore these warning signs. We live in a society where food is used for pleasure or entertainment. But its main purpose to the body is for energy fuel. Just like you select what kind of fuel to put in your car you need to select what food you put in your body to derive its energy. The type of energy will determine the mileage or distance the car runs. A food that does not provide the energy needed is just useless to the body. It is that simple.
I spent years with a chronic gastritis and I could not find any relief. I was operating on a low energy level that manifested itself into a general malaise, fatigue and insomnia. The daily work stress was not helping either. I thought that was the life now. Eating was no longer fun due to the aftermath. I had lost interest in food altogether until I made a drastic change in my diet.
I eliminated all dairy or animal products including meat, cheese and milk. Within a few months I noticed the changes in my energy level. I lost weight. I feel better and stronger. My eyes look clearer. I sleep better. My skin glows and looks healthier than ever. I was sick and I didn't know it. The food I was eating was the venom killing me slowly.


You know how Google Calendar lets you have multiple calendars, i.e. weather, etc?
There is an obscure page that looks like it's designed for mobile devices that lets one choose which calendars to sync with Google Sync.
I can't seem to dig it up. Does anyone know where it is?
You should be able to do all the changes from: m.google.com/sync directly from your device.
On the upside, you no longer have to be on a mobile device.


You'll look as sweet as a cookie in this Cookie Monster Plush Tween Costume. Satisfy your sweet tooth in this blue plush dress featuring a cute cookie accent. Your costume wouldn't be complete without a headpiece featuring Cookie Monster's iconic googly eyes.


text:
Q: Postgres cursors behaviour I am trying to understand a PostgreSQL behaviour.
In my application I am using a Postgres (AWS Aurora to be precise) database through ODBC.
I am not specifying "with hold" on the cursors used but I see in the logs that most of them are 'idle in transaction', although there is no transaction, there is just a 'select'. The statements look like:
RELEASE _EXEC_SVP_05A0C768;SAVEPOINT _EXEC_SVP_05A0C768;declare "SQL_CUR05A0E3F8" cursor with hold for select u_selskap,u_kodetyp,u_tekst from u_hrm.u_stilltyp where u_selskap=50 and u_kodetyp='T';fetch 100 in "SQL_CUR05A0E3F8"

In PgAdmin we can see how the statements are displayed.

The idling cursors seem to keep locks on the database hence subsequent queries cannot be run.
I am not able to understand why these cursors are in a transaction and which part of the chain is introducing the 'with hold' as the default creation mode is 'without hold'(as seen here: https://www.postgresql.org/docs/11/sql-declare.html).
Can it be a PostgreSQL ODBC issue?

redpajama_set_name: RedPajamaStackExchange


text:
PHO Alliance outlines seven-step plan for a sustainable health system
Written by Kate McDonald on 20 February 2015 .
The PHO Alliance has outlined seven policies that it believes need to be introduced to help New Zealand's health sector overcome the challenges it will increasingly face.
Three actions concern reprioritisation of funding and services, including a commitment to supporting IT systems that directly underpin patient-centred integrated care, and four are aimed at removing barriers to access and eliminating "perverse incentives".
The PHO Alliance, which represents 21 primary health organisations, has presented its A time to act publication to the Minister for Health, Jonathan Coleman, saying the seven actions will help sustain the New Zealand health service for future generations.
PHO Alliance chairman John Ayling said the NZ health service was one of the best in the world, "but we have an ageing population living with more long-term conditions, increasing incidence of obesity, the challenge of technological advancement and rising public expectations, all of which are combining to place unsustainable pressures on hospitals, general practices and aged care providers."
Manaia Health PHO chairman Andrew Miller said the actions asked some challenging questions.
"Whilst we have a great primary care system, we know we cannot meet rising demand using the same models of care and provision as we do now," Dr Miller said.
"We need to address those areas of the current system which are failing to keep pace with demand and are failing to address the unacceptable inequalities being experienced by our most vulnerable and high-needs patients."
The publication says a focus on health promotion, prevention and health literacy is critical to address the wider determinants of population health and a fundamental shift is urgently needed in the way healthcare is delivered.
"If we get the care right, it will be right for the patient as well as the system, then sustainability will follow," it says.
The first action is to prioritise more services and funding to those most in need, especially Maori, Pacific Islanders and those New Zealanders living in the most deprived communities, by changing the capitation system and better targeting funding from the "myriad funding silos" that span Ministry of Health programs.
"The current system of universal capitation funding is failing those who need it most and the formula for allocating health dollars across both primary and secondary care services needs an overhaul to be targeted for best effect," it says.
"The big advances in reducing inequalities come when bold decisions are made about funding priorities."
It also wants the government to abandon health targets that have no evidence of improvements to patient outcomes, such as setting maximum waiting times for emergency departments and DHB targets for the number of patients receiving elective procedures. This, the PHO Alliance argues, does nothing for preventative health or for finding alternatives to secondary care.
It also wants to put integrated IT platforms in place, supporting the vision of the National Health IT Board but arguing more needs to be done to make it a reality.
"To achieve high quality healthcare and improve patient safety, New Zealanders need a core set of personal health information available electronically to them and their treatment providers regardless of the setting as they access health services," it says.
"We need to find information solutions which allow patient-centred integrated care. These systems need to be interoperable, share data and be accessible to not only health providers but patients.
"For this to occur there has to be strong national and regional leadership to make unified decisions about how to progress towards the NHITB vision statement. This should be driven by the needs of patients and clinicians, not by IT vendors."
Perverse incentives
In terms of removing the barriers to access and eliminating perverse incentives, it argues that all primary care consultations must be made more affordable.
"To allow patients to attend ED with its immediate access to diagnostics and its high-tech facilities with no co-payment, yet have financial barriers to see their GP, is perverse," it says.
"The current Very Low Cost Access (VLCA) capitation funding formula is failing. There are 590,000 non-high needs patients receiving the benefit of VLCA funding. We need to have a funding formula that is targeted to the need of the individual patient."
It also wants to improve multidisciplinary care by allowing patients direct access to allied health professionals and free up GP time for more complex patients.
"Once again, we believe removing the funding barriers and better co-ordinating access to all our primary care health professionals will not only improve outcomes but also save money as we optimise the skill mix for the primary care workload and reduce demand upon high cost secondary care services and ED."
It wants to remove barriers to diagnostic services by allowing primary care teams to better use diagnostic services that are traditionally the domain of secondary care.
"Failure to provide access to the full suite of diagnostic services to our vital primary care workforce results in delays to diagnosis, anxiety for patients, duplication of costs through outpatient appointments and an overwhelming failure to provide the right care at the right time in the right place."
The seventh action is to challenge what the PHO Alliance calls the "conflicted relationships" that DHBs have as both controllers of funds and managers of hospitals.
"In New Zealand, we feel investment in primary care has continually been eroded over recent years rather than increased," it says.
"We believe this will not change whilst we have financial pressures at the same time that DHBs have the unenviable task of controlling those investment decisions locally and simultaneously being held robustly and publicly to account for the performance of secondary care acute hospitals.
"Very few directors or executives the world over could rationally increase the risk to their own bottom line by investing in another sector's growth and capacity.
"We need to look again at the purchaser/provider split without creating an industry of accountants and contract managers unwittingly diverting health funds away from real patient care."
Mr Ayling said some of the proposed actions would require bravery to implement given that they confront an established way of thinking and established practice.
However, "improved patient outcomes must stand above existing arrangements and organisational barriers," he said.
"Now is the time to act for those communities and families most at need."
The publication (PDF) is available from the PHO Alliance website.
Posted in New Zealand eHealth

redpajama_set_name: RedPajamaCommonCrawl


We deliver solutions that simplify how PMCs and HOAs connect, operate, and secure modern communities. With a focus on mobile-first technology, our tools make it easy for community leaders and homeowners to communicate, collaborate, and drive a more connected community. With our powerful integration with TOPS[ONE] you can manage all of your association accounts from a single login or our mobile app, get paid on time, and eliminate fraud and liabilities. By removing administrative burdens and increasing adoption rates, our advanced payments solution will fundamentally change the way payments are handled in the HOA space.
More than 23,000 communities nationwide - that's 4.4 million homeowners - are covered by FRONTSTEPS services.


If your dog has never tried SmartBarf before, please order a sample bag as we cannot accept opened bags back and returns are at the customer's expense unless the product is faulty.
At the suggestion of our customers, we are now shipping out multi-pack offers (any purchase of 7 or more packs) in dark bags without the cardboard display packaging. This does not affect stockists who will continue to receive packs in boxes as before.
The couriers we currently use are DPD for all orders apart from the samples and 1-pack orders and those to certain locations (Royal Mail). If you have any issues with our preferred courier, mention it on the order form (order notes).
SmartBarf™ is a 100% natural source of minerals, vitamins and nutrients from over 50 different seeds, nuts, fruits, berries, vegetables, pulses and herbs. All the ingredients are carefully selected for maximum nutritional benefit, to create a healthy balanced diet for your dog.
SmartBarf™ is already mixed and packaged, and has a long shelf life, all for your convenience and to save you time and money. Just add water and serve with meat.
For a full list of Ingredients – click here.


Being called to father's office sometimes happened.
Unless he needed us for something, my constantly working father practically ignored us. When he did have some business or another with us, then he would summon us in this fashion. It was never about us, just his one-sided problems.
You could say I was a little annoyed. I wanted to try saying the words on my mind: 'Is a daughter just a useful tool to you!' but I never did.
Unfortunately, I knew why I was being brought in today.
That's why I declared this as soon as I entered the room.
Seems my father was disappointed I had stolen his thunder. Of course a genius such as I would not misunderstand such a simple problem. I was quite taken aback by his words. 'Again'? Just what did he mean by 'again'? That makes it sound like I'm always misunderstanding things.
I held a composed smile when faced with his eyes that seemed to be saying 'my daughter is spouting nonsensical things again'.
I'm sure the surprise on his face was caused by the fact I had guessed his intentions so perfectly. The one misunderstanding is you father, underestimating me that far. I thought and let out a chuckle at his astonished reaction.
While raising my voice I slapped down the list I had compiled on the desk in front of my father. Listed there were all the people who had dared twist and spread lies about me in high society. Still in shock my father quietly picked up the list.
As he surveyed the list that was the fruit of my efforts my father's features seemed to sadden.
I didn't understand the situation as my father's words trailed off midway.
Somehow his reaction seems off. Even though he should be shaking with amazement at my genius, his expression is the exact opposite. It almost went so far as to be an expression of exasperation.
Sighing my father placed the list back on the table.
Even though I thought he was just joking as a sore loser, he corrected me immediately. It seems his call today is truly about a separate matter.
After all my earlier confidence I felt a little off put.
The originator of the information was guests of the party, in other words children. It was unlikely that such young children, who had not even debuted into high society yet would be able to spread such rumours.
In other words, the people spreading the rumours are the parents.
Since it's adults spreading the malicious rumours, there is no way it's just harmless gossip. Nine cases out of ten, their aim was to ruin the Noir family by attacking the reputation of the only heir. it could also incidentally be a diversion. As long as it's speaking ill of the Noire family, those people are fine with anything.
Perhaps out of agreement, my father nodded easily.
My pride will never forgive others sullying my name with rumors. My father ignored my interruption and continued.
I had not expected her name to come out here.
If their intention was truly to topple the family, they would not be spreading flattering rumors about the adopted daughter. But with my little sister being such an absolute angel isn't it obvious that rumors of her cuteness would spread? It was inevitable that such an angel wouldn't be touched by such evil intent.
If it's about Michelie's birth, then it would be about her mother being the younger sister of the current King, Her Royal Highness, Evelia Edward.
But how was that even connected to the current rumors? All in all there was really no connection that I could see.
My father would give a clear answer even though I was so confused. Instead my father held out the envelope that had been sitting on his desk.
He had closed the essential part of the conversation while dodging actually answering anything. I was left only being able to grasp at straws. I couldn't read my father's face at all.
Looking at the face I was convinced to let the matter drop.
It was the face of a noble who had clearly drawn the line between his personal life and official role. Right now, this man was not my father, but rather an official with a job to do. Even if I tried to pry it out of him it would be useless.
Although I was frustrated at all the secrets, I still gasped when I saw the wax seal on the letter. My eyes couldn't get any wider.
The seal on the letter was without a doubt the royal crest. Of course I also had an idea of who the sender might be. Expecting the letter to be from Charles my face turned bitter.
I thought for sure that Charles had grown impatient and used his power to summon me, but it seems I was wrong. I had been so convinced it was from Charles that I hadn't even confirmed the sender.
Before I could check the envelope for the name of the sender, my father said it out loud.
Although this conversation had been nothing like I'd thought, I will still shocked at the name he chose to throw out at the end. Upon hearing the name of the sender my thoughts froze.
However, being a genius that I was, I quickly regained my thoughts. Slowly I raised my face as I understood the truth of what was happening.
Even though it was the first expected answer in this whole conversation, I couldn't be happy about it. My shoulders fell at his harsh dismissal.
As the first prince of the country, he was next in line to succeed the throne. Along with Charles and Leon, he made up the last person of the three conquerable targets of Labyrinth Destiny.
And above all there is one thing that I had to mention.
If I was to describe his character in the game…It was one that he was prejudiced from the very bottom of his heart.
The father's attitude to his daughter reminds me of Observation Record of a Self-Proclaimed Villainess.


If you went to the Manila International Auto Show in April, you likely saw the centerpiece of Suzuki Philippines' booth at the annual event: the all-new Dzire subcompact (or mini?) sedan. Previously called the Swift Dzire as an acknowledgment of the fact that it was based on the Swift hatchback, the car is officially its own model now, dropping its trunkless sibling's moniker from its name. At the time, we said that the launch date could take place within a month.
Today, Suzuki's local distributor gave the motoring media a chance to drive the Dzire at Batangas Racing Circuit ahead of the small sedan's formal market introduction on June 27.
Imported from India (where Maruti Suzuki mass-produces it), the new Dzire measures 3,995mm long, 1,735mm wide and 1,515mm tall, with a wheelbase of 2,450mm. By comparison, the Mitsubishi Mirage G4 has the following dimensions: 4,245mm in length, 1,670mm in width and 1,515mm in height, with a 2,550mm wheelbase. While the Dzire is shorter, it is also wider from side to side, contributing to just a little more space girth-wise.
The new-generation Dzire is based on Suzuki's new Heartect platform that is touted as having lighter weight, higher rigidity and better energy absorption in the event of a collision. You might recall that the recently unveiled second-generation Ertiga is also already based on this platform.
One thing you'll immediately appreciate about this latest Dzire version is that it is significantly better-looking than its predecessor, which was essentially just a hatchback with an awkwardly grafted trunk at the back. The new model represents a more homogeneous styling execution, so that it appears to be a true sedan.
Under the hood of the new Dzire is a carried-over 1.2-liter four-cylinder gasoline engine that has reportedly been tweaked to achieve a higher compression ratio and better fuel efficiency. Rated at 82hp and 113Nm, this powerplant is mated to either a five-speed manual shifter or a five-speed Auto Gear Shift transmission. Suzuki Philippines assistant general manager Cecil Capacete made sure to emphasize the number of cylinders as this obviously trumps the three-pot mill shoehorned into the Mirage G4.
The whole point of the four-lap test drive was for Suzuki to flaunt the new Dzire's Auto Gear Shift transmission, which is basically an automated manual gearbox that features an electro-hydraulic actuator to automatically engage the clutch and the gearshift. In Drive mode, the AGS behaves just like a regular automatic. In Manual mode, it delegates shifting duties to the driver—you nudge it forward to downshift and you tug it toward you to upshift. As with any newfangled gizmo, the AGS takes some getting used to. Perhaps why we were given four laps each. There is some trick required in avoiding shift shocks, a simple action that involves releasing the gas pedal before you change gears. The process felt confusing at first, but quickly became intuitive as the corners piled up.
The new Suzuki Dzire will be offered in two variants: the P638,000 GL MT and the P698,000 GL+ AGS. Yes, it undercuts the longer but narrower and lower Volkswagen Santana (P686,000). We have a feeling Suzuki Philippines put together the media preview to sway countless consumers seriously tempted by the entry-level VW. This should be an interesting fight.
New Suzuki Swift: Want the Thai or the Indian version?
With revised taxes, how are car prices affected?


text:
Jahsonic (Talk | contribs)
← Previous diff Revision as of 07:49, 16 February 2015
+ {| class="toccolours" style="float: left; margin-left: 1em; margin-right: 2em; font-size: 85%; background:#c6dbf7; color:black; width:30em; max-width: 40%;" cellspacing="5"
+ | style="text-align: left;" |
+ It is [[Camille Paglia]]'s central thesis that in the [[20th century]] (which she calls the Age of Hollywood) [[pagan]] [[popular culture]] overtook and vanquished the [[high arts]]. Thanks to advances in technology, pop became a universal language, as catholic in its reach as the medieval church. Once pop art embraced commercial iconography, the [[avant-garde was dead]].
+ |}
[[Image:Hollywood.jpg|thumb|right|200px|[[Hollywood]] is iconic for the [[mainstream]]]] [[Image:Hollywood.jpg|thumb|right|200px|[[Hollywood]] is iconic for the [[mainstream]]]]
{{Template}} {{Template}}
*[[NC-17]] *[[NC-17]]
*[[Peg Entwistle]] (1908 – 1932), an English actress who gained notoriety after her suicide at the age of 24 by leaping off of the Hollywood Sign. *[[Peg Entwistle]] (1908 – 1932), an English actress who gained notoriety after her suicide at the age of 24 by leaping off of the Hollywood Sign.
+ ==Namesakes==
+ *''[[Hollywood Boulevard]]'' by Joe Dante
+ *''[[Hollywood Babylon]]'' (1959) by Kenneth Anger
== References == == References ==
It is Camille Paglia's central thesis that in the 20th century (which she calls the Age of Hollywood) pagan popular culture overtook and vanquished the high arts. Thanks to advances in technology, pop became a universal language, as catholic in its reach as the medieval church. Once pop art embraced commercial iconography, the avant-garde was dead.
Hollywood is iconic for the mainstream
Hollywood is a district in Los Angeles, California, U.S.A.. Due to its fame and cultural identity as the historical centre of movie studios and stars, the word "Hollywood" is often used as a metonym for the American film and television industry. In fact, one can safely regard Hollywood, or the American film industry, as the pinnacle of mainstream cinema.
Hollywood Sign
American mythology
20th century mythology
Peg Entwistle (1908 – 1932), an English actress who gained notoriety after her suicide at the age of 24 by leaping off of the Hollywood Sign.
Hollywood Boulevard by Joe Dante
Hollywood Babylon (1959) by Kenneth Anger
Easy Riders, Raging Bulls, 1998, Peter Biskind, on New Hollywood
Hollywood Babylon, 1959, a book by Kenneth Anger about the sordid scandals of Hollywood denizens from the 1900s to the 1950s
The Hollywood Hallucination, 1944, Parker Tyler
The Life and Death of 9413: a Hollywood Extra, 1928, short silent film written by Robert Florey and directed by Florey and Slavko Vorkapich
The Celluloid Closet, 1981, a book by Vito Russo on how Hollywood films, have portrayed gay, lesbian, bisexual and transgender characters
Unless indicated otherwise, the text in this article is either based on Wikipedia article "Hollywood" or another language Wikipedia page thereof used under the terms of the GNU Free Documentation License; or on original research by Jahsonic and friends. See Art and Popular Culture's copyright notice.
Retrieved from "http://www.artandpopularculture.com/Hollywood"


There's one thing that a press release needs. It's more important than any other element in a press release, and that's a story. Your press release needs a purpose, a reason to write a press release. Your press release marketing strategy may require you to write consistent press releases. That means you need to be able to find stories. Let's take a look at 24 questions to help you accomplish that on a regular basis.
1.	What challenges did the company overcome, either in its early days or at some other point in time?
2.	How has your company impacted others in a positive way? Think about your community and your customers.
3.	Is your company invested in your community and if so, how? What have you done to improve the community or get involved?
4.	Has the industry or market changed and impacted your company in some way?
5.	Have you gotten any new significant or interesting customers?
6.	How did your products come into being? What is their story? Why were some products or services created?
7.	Are there any famous people who are associated with your product or your company?
8.	What's unique about your product that sets it apart from the competition?
9.	Is your brand associated with any special national or regional events or holidays?
10.	Has your brand changed over time? Is there a story related to these changes?
11.	What is the story behind your logo?
12.	Does your brand have an official spokesperson? If so, what is the reason for this relationship and what stories are associated with it?
13.	Is your brand associated with a cause? If so, what is the story behind this relationship?
14.	What did your company founders do? What inspired them? Can your employees or customers relate to this history?
15.	Are any of your employees members of the military, volunteer firemen or other first responders?
16.	Are any of your employees members associated with a special cause? If so, what is it and why is it important?
17.	Has your company done any research lately? Can your company introduce exciting new statistics or information?
18.	Have you hired someone new recently? Or has an important person left your company?
19.	Have any of your employees done something special for the community?
20.	Has your company expanded?
21.	Has your company entered a new market, started a new business or shifted the focus of the business?
22.	Has your company released any new products or services?
23.	What are your employees like? Do they do anything special or interesting? Consider doing a series of employee profiles.
24.	What is the personal story of your senior management? Have they done anything that provides lessons or information for your customers?
These 24 questions should be sufficient to fuel a number of brainstorming sessions. You may have enough story and press release ideas to fill several years' worth of press release marketing. Newsworthy content is what the media is drawn to, and stories attract attention. Combine the two to create attention grabbing and business building releases.


Call of Duty Modern Warfare 2 will come in three different packages: normal, Hardened Edition and Prestige Edition. The Prestige Edition comes with working night-vision goggles, steel game case, art book, custom molded plastic head (for your night-vision goggles) and a voucher code for downloading the original Call of Duty.
The Hardened Edition will be selling for $80 and the Prestige Edition for $149. Check out the unboxing of the Prestige Edition below.


text:
Daytime Ratings: 'The Talk' Continues Season-Long Audience Collapse While 'The View' Soars
Check Out Emme Rylan's New Hubby!
byKambra Clifford
Disney/ABC Television Group (Top); Sony Pictures Television (Bottom)
TeleNext Media, Inc.
Emme Rylan may be between soaps at the moment (having recently left her role as "The Young and the Restless'" Abby Newman in order to play "General Hospital's" Lulu Spencer), but you can still catch her onscreen — if you're quick!
Robert Voets/CBS
The Pennsylvania native will be appearing on tonight's episode of "CSI: Crime Scene Investigation," and earlier today, she sent out a tweet revealing that she plays the wife of David Phillips (played by David Berman), who serves as the CSI team's lead researcher. "I'm in the first and the last scene of the episode," she says. "Be on the look out!"
You can catch the episode tonight on CBS at 10:00 PM EST, or you can download the new CBS app to access the episode after its network airdate.
Cedric 'The Entertainer' Replacing Meredith Vieira on 'Millionaire'; 'General Hospital' Related Episodes Scheduled
What's Molly Burnett Doing With A 'Ladies' Man'?
Donna Mills Announces 'GH' Airdate!
Ever since it was announced that former "Knots Landing" actress Donna Mills is headed to "General Hospital," tongues have been wagging regarding character details and a first airdate. And while the Illinois native has yet to reveal any details about her new alter ego, she has announced when fans can expect her to make her Port Charles debut!
'AMC's' Ambyr Childers Expecting Second Child
Hot on the heels of the news that both "All My Children" stars Eden Riegel (Bianca Montgomery) and Paula Garces (Lea Marquez) are expecting babies comes the news that Ambyr Childers (ex-Colby Chandler) is pregnant, too!
'Katie' Premiere Date Set, Last Episode of 'GH' Too?
On Thursday, March 15, 2012, the Disney/ABC Television Stations Group released a press release (see below) informing stations across the country of its plans to launch the long anticipated talk show "Katie," hosted by news personality Katie Couric, on Monday, September 10. What wasn't stated was "General Hospital," which currently occupies the planned 3:00 PM ET/2:00 PT/CT timeslot for Couric's talk show, is likely to air its final episode in that timeslot on Friday, September 7. As we all know ABC is scheduled to turn over the timeslot to its affiliates, which will allow them to air various programming across their stations, including the option of airing "Katie."
'DAYS' Star Blake Berris Portrays Drug Dealer In 'Meth Head'
Blake Berris may be best known for his role as Nick Fallon on "Days of our Lives," but the young actor has been busy trying to make a name for himself via other avenues, as well. He's recently made headway in the indie film circuit, and his latest project is Jane Clark's "Meth Head," a story about a man named Kyle, played by Lukas Haas ("Lincoln," "Inception," and "Brick"), who loses control of his life after becoming addicted to methamphetamines.


text:
E.A.Poets Approved/Bands We Like – Javier Carrillo
our "E.A.Poets Approved/Bands We Like" section is dedicated to Javier Carrillo. I discovered this Artist while shuffling songs on Spotify and I liked the easy vibes of his music.
The delicate atmospheres created by this artist's music immediately put me in a good mood.
Music that recalls those days when the rain has just ended and the sun begins to filter through the clouds and the people begin to leave the house again to have fun.
This airy and ethereal sound is emphasized by the vocal timbre of this talented artist who, with his light and sweet voice, accompanies you on a beautiful sonic journey.
The production is very professional and in particular, the mix that leaves the right space for each sound element and the song breathes well.
A nice discovery that I recommend to all lovers of light and modern pop-rock that by the way is never banal.
No Sorrow is Javier Carrillo's Single Out Now!
Sweet and Refined!
Javier Carrillo (Javi) is a singer-songwriter, multi instrumentalist, and producer based in Dublin, Ireland.
Born in Peru, his biggest influences were the alternative rock from the 80s and 90s, and the "trova" music popular in South America. He also lived in Italy for a few years where he studied and polished his piano and guitar skills.
Motivated by its music scene, he moves to Ireland where he joins a songwriting programme at BIMM which also connects him with local bands like Darwin's Daughter and NIMF with whom he collaborates as guitarist and drummer.
Find Javier Carrillo Here:
80's music rock 80's rock 80s music magazines 80s rock album covers 80s rock and roll hits 90s music magazines album covers classic best classic rock vinyl best rock magazine classic guitar rock classic of rock classic rock albums classic rock art digital music magazine funko pop hard rock cd Javier Carrillo mag band metal and rock music metal rock No Sorrow rock music magazine


text:
<!DOCTYPE html>

    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <meta name="generator" content="rustdoc">
    <meta name="description" content="API documentation for the Rust `Protocol` trait in crate `nemo`.">
    <meta name="keywords" content="rust, rustlang, rust-lang, Protocol">

    <title>nemo::Protocol - Rust</title>

    <link rel="stylesheet" type="text/css" href="../main.css">

nemo

        <form class="search-form js-only">

                <input class="search-input" name="search"
                       autocomplete="off"
                       placeholder="Click or press 'S' to search, '?' for more options…"
                       type="search">

        </form>

Trait nemo::<wbr>Protocol

                [−]

        [src]

pub trait Protocol {
    type Initial: SessionType;
}

A Protocol describes the underlying protocol, including the "initial" session
type. Handlers are defined over concrete Protocols to implement the behavior
of a protocol in a given SessionType.

Associated Types

type Initial: SessionType

Implementors

Help

Keyboard Shortcuts

?

Show this help dialog

S

Focus the search field

⇤

Move up in search results

⇥

Move down in search results

⏎

Go to active search result

Search Tricks

                    Prefix searches with a type followed by a colon (e.g.
                    fn:) to restrict the search to a given type.

                    Accepted types are: fn, mod,
                    struct, enum,
                    trait, type, macro,
                    and const.

                    Search functions by type signature (e.g.
                    vec -> usize)

redpajama_set_name: RedPajamaGithub


Join the Wyoming Retail Association!
Wyoming's only organization dedicated exclusively to the interests of retail businesses.
Our efforts ensure that your voice is heard in the Wyoming Legislature and Congress.
You can now save up to $6,000 per year on your sale tax remittance due to WRA efforts in the passage of the vendors compensation bill.
WRA advocates for favorable business laws, lower taxes and less government.
We work to protect you from unfavorable unemployment insurance and workers compensation rules that can cost you money.
WRA Office & WRA Website - Consider us an extension of your staff; save time and money by using our resources to answer any of your questions.
WRA Newsletters - Featuring the latest industry news, trends, best practices, marketing ideas, legislative updates, and upcoming events to help you operate more efficiently.
Free membership and access to a wealth of information from the National Retail Federation.
Free Wyoming Labor Law CD - An extensive resource explaining the details of Wyoming labor laws.
Receive significant discounts on select FedEx® services and other freight shipments with leading national and regional carriers on less-than-truckload shipments (LTL).
Save money when filling prescriptions with the free WellDyneRX discount card. A great incentive for your employees and even their families!
Advertise on GoExplore WYOMING! the Cowboy State's Smartphone App with special WRA member pricing. This professional turn-key solution will help you break into mobile marketing.
Business solutions at a reduced cost from Avitus Group. Let them handle payroll, bookkeeping, human resources, marketing, and more, leaving you time to focus on your business.
Receive the most competitive rates with Heartland Payment Systems - a company you can trust for your credit card processing.
Pioneer Printing, based in Cheyenne makes it possible to choose a selection of beautiful papers and quality products without compromising your budget. WRA members save an average of 10%.
Office Supply discounts with Staples - Quality products delivered to your door with an average savings of 60%.
For more information contact Tom Miller at 307-634-7768, ext. 204.


Rich Tone Music are a main Fender Dealer and we stock a large range of Fender Bass Guitars to cover numerous price points.
When it comes to bass guitars, nothing is quite as iconic as a Fender Precision or a Fender Jazz bass. We stock numerous example of each type of Fender Bass Guitar that are all available to buy online or in store.


Atlanta, GA recording artist Oren Major finally drops off his much anticipated sophomore album, "Black Privilege." Oren's concept for the project was essentially centered around hope, encouraging everyone to embrace their culture and take pride in everything - the good, the bad, and the ugly. Despite being named "Black Privilege," he says you don't need to be black to understand the message and that anybody who has endured hardships and persevered can relate. He aims to mix the dense and powerful lyricism he is known for with soulful, catchy tunes, describing it as "sounds you can feel." The goal of the album was to deliver a message to the listener that no matter your situation, you are solely in control of your life and your outcome.


Evidence of a decline in long-run growth is accumulating. However, many important questions remain unanswered. The analysis in column employs recent econometric techniques to provide an answer to some of the pertinent questions. The findings indicate that the weakness of the current recovery in the G7 is associated with a decline in the long-run growth rate of labour productivity.
The slow pace of the recovery from the Great Recession of 2007-2009 has prompted questions about whether the long-run growth rate of GDP in advanced economies is lower now than it has been on average over the past decades, as famously articulated by Robert Gordon and Lawrence Summers (Teulings and Baldwin 2014).
Indeed, evidence of a decline in long-run growth is accumulating. The application of popular statistical tests for structural breaks suggests that evidence in favour of a shift in the mean of US GDP growth has been building up and has recently become significant at the 5% level (Figure 1). The Bai and Perron (1998) test can be used to detect the most likely date at which the break occurred and this is found in the early part of the 2000's.
Note: The solid gray and blue lines are the values of the test statistics obtained from sequentially re-applying the Nyblom (1989) and Bai and Perron (1998) tests in real time as new National Accounts vintages are being published. In both cases, the sample starts in 1960 and the test is re-applied for every new data release occurring after the beginning of 2000. The dotted line plots the 5% critical value of the test, while the dashed line plots the 10% critical value.
Orphanides (2003) emphasised how real-time misperceptions about the long-run growth of the economy can play a large role in monetary policy mistakes.
Even small changes in assumptions about the long-run growth rate of output can have large implications on fiscal sustainability calculations.
These points highlight the importance of accurately assessing the current long-run growth rate in a timely manner.
In this respect, Figure 1 also highlights that the strategy of applying conventional tests in real-time is therefore not satisfactory for the purpose of real-time decision making. In fact, the detection of change in the mean of GDP growth can arrive with substantial delay. For instance, the Bai and Perron (1998) test would not detect the early 2000s break at a 5% significance level until the summer of 2014, with almost 15 years of delay!
Since the seminal contributions of Evans (2005) and Giannone et al. (2008), dynamic factor models (DFMs) have become the standard tool to track GDP. These models can incorporate a large amount of information and of exploit high frequency data to make inference about short-term developments in economic activity.
In a recent paper (Antolin-Diaz et al. 2015), we extend this framework by allowing for gradual changes in the mean and the variance of real output growth. By incorporating a large number of economic activity indicators, dynamic factor models are capable of precisely estimating the cyclical comovements in macroeconomic data in a real-time setting. Our extended model exploits this to track changes in the long-run growth rate of GDP in real time, separating them from their cyclical counterpart.
Note: Panel (a) plots the posterior median (solid red), together with the 68% and 90% (dashed blue) posterior credible intervals of long-run GDP growth. The gray circles are the CBO's estimate of potential growth. Shaded areas represent NBER recessions. In Panel (b), the solid gray line is the filtered estimate of the long-run GDP growth rate, using the vintage of National Accounts available as of mid-2014. The blue diamonds represent the real-time mean forecast from the Livingston Survey of Professional Forecasters of the average GDP growth rate for the subsequent 10 years.
Since in-sample results obtained with revised data often underestimate the uncertainty faced by policymakers in real time, we repeat the exercise using real-time vintages of data. By the summer of 2011, the model would have concluded that a significant decline in long-run growth was behind the slow recovery, well before the structural break tests became conclusive. Moreover, explicitly taking into account movements in long-run growth significantly improves the accuracy of both point and density 'nowcasts' of US GDP.
What lies behind the slowdown in long-run growth?
Next, we extend our model in order to disentangle the drivers of secular fluctuations of GDP growth. In our framework, by adding information about aggregate hours worked, long-run growth can be decomposed into the labour productivity and labour input components.
The results of this decomposition exercise point to a slowdown in labour productivity as the main driver of recent weakness in US GDP growth, in line with the analysis of Fernald (2014).
Note: The figure plots the posterior median (red), together with the 68% and 90% (dashed blue) posterior credible intervals of long-run GDP growth and the posterior median of both long-run labour productivity growth and long-run total hours growth (solid blue and dashed grey lines).
Applying the model to other advanced economies, we provide evidence that the weakening in labour productivity appears to be a global phenomenon.
Note: Panel (a) displays the posterior median of long-run labour productivity across advanced economies. Panel (b) plots the corresponding estimates of long-run total hours worked. In both panels, 'Euro Area' represents a weighted average of Germany, Italy and France.
Moreover, the results of our decomposition exercise indicate that after using the dynamic factor model to remove business-cycle variation in hours and output, the decline in long-run GDP growth that has been observed in the advanced economies since the early 2000s is according to our model entirely accounted for by a decline in long-run productivity growth. Finally, it is interesting to note for the G7 economies long-run productivity growth appears to converge in the cross-section, with the latest estimates suggesting convergence towards a low growth rate, between 0.5 to 1%.
Using a state-of-the-art econometric technique we provide evidence for a slowdown in labour productivity as the main driver of weak global growth in recent years, which supports the narrative of Fernald (2014), also for countries other than the US.
We have demonstrated that long-run movements in labour input and labour productivity are an important feature of the data that can be successfully modelled within the dynamic factor model framework. Our econometric framework remains agnostic about the deep structure of the economy. In principle, these low-frequency movements could be influenced both by demand and supply factors. Disentangling the two is a promising avenue for further research.
 See Banbura et al. (2012) for a survey of the literature on nowcasting GDP.
 Similar evidence for emerging economies has been recently presented by Pritchett and Summers (2014). Their evidence refers to convergence of overall GDP growth rates, whereas ours indicates that convergence in productivity growth appears to be the dominant source of convergence.
Antolin-Diaz, J, T Drechsel, I and Petrella (2015), "Following the Trend: Tracking GDP when long-run growth is uncertain", CEPR Discussion Papers 10272, C.E.P.R. Discussion Papers.
Bai, J and P Perron (1998), "Estimating and testing linear models with multiple structural changes", Econometrica, 66(1):47-68.
Banbura, M, D Giannone, M Modugno, and L Reichlin (2012), "Now-casting and the real-time data flow", Working Papers ECARES 2012-026, ULB - Universite Libre de Bruxelles.
Evans, M D D (2005), "Where Are We Now? Real-Time Estimates of the Macroeconomy", International Journal of Central Banking, 1(2).
Fernald, J (2014), "Productivity and potential output before, during, and after the great recession", NBER Macroeconomics Annual 2014, 29.
Giannone, D, L Reichlin, and D Small (2008), "Nowcasting: The real-time informational content of macroeconomic data", Journal of Monetary Economics, 55(4):665-676.
Gordon, R J (2014), "The Demise of U.S. Economic Growth: Restatement, Rebuttal, and Reflections", NBER Working Papers 19895.
Orphanides, A (2003), "The quest for prosperity without ination", Journal of Monetary Economics, 50(3):633-663.
Pritchett, L and L H Summers (2014), "Asiaphoria Meets Regression to the Mean", NBER Working Papers 20573.
Summers, L (2014), "Secular stagnation", IMF Economic Forum: Policy Responses to Crises, speech delivered at the IMF Annual Research Conference, November 8th.
Teuling, C and R Baldwin (2014), "Secular stagnation: Facts, causes, and cures – a new Vox eBook", VoxEU.org, 10 September.


- Desert Survival Reality Tv Show Food Storage Emergency 7 Day Shenandoah Junction Weather Water is obviously the most crucial thing you may want to store replenish defend. Oktober veröffentlicht. ★ ★ ★ Xm855 Shtf - Guide to Emergency Survival in America.
The following is a listing of officially released works by the English band Porcupine Tree. Watch breaking news videos original video clips on c 20, · David Bowie nome artístico de David Robert Jones, viral videos , Brixton, Londres especificamente no arch the history of over 347 billion web pages on the Internet. Where are they now? Xm porcupine tree download.
This page last updated:. Modern 3 Emp Technology Holdings Llc Snacks really are essential disaster survival foods Download with Google Download with Facebook or download with email. Die erste Porcupine- Tree- Konzert- DVD Arriving Somewhere.
Auf der folgenden Tour wurde in der ersten Hälfte der Shows 50 Minuten neues Material gespielt wobei sich im Januar herausstellte dass dieses Material auf dem kommenden Album veröffentlicht wird. Xm porcupine tree download. ★ ★ ★ Emp Technology Holdings Llc - Guide to Emergency Survival in America. Dans un premier temps une expérimentation en solo du guitariste le groupe se composait, chanteur et compositeur Steven Wilson, durant ses dernières années d' existence de quatre musiciens.
This is the second Porcupine Tree Sampler ( this one dated ) this one contains tracks collected from various solo projects , like the previous one released in non- PT projects that the members of the band were involved in during the time. 网易云音乐是一款专注于发现与分享的音乐产品, 依托专业音乐人、 dj、 好友推荐及社交功能, 为用户打造全新的音乐生活。. - Yes and projects with multiple Yesmen.
@ XM855 SHTF @ Watch FREE Video Now! Porcupine Tree est un groupe de rock et metal progressif britannique originaire de Hemel Hempstead en Angleterre. Hattic as a Sino- Caucasian language [ UF 41, – ]. Die Tour wurde von der schwedischen Band Paatos, Oceansize und ProjeKt 6. While Satellite may be a new name in the prog field Sunset' their debut album, this is a band that have been formed out of one of Poland' s most well- known , best- loved prog acts Collage. Emergency Food Storage 15 Year Plan Xm855 Shtf The first decision that in order to be be made when considering survival food. L' album le plus vendu en date est Deadwing, publié en [ 2]. @ EMP TECHNOLOGY HOLDINGS LLC @ Watch FREE Video Now! # Linux factpack # 1999- Dec- 29 # Author: Kimmie Dicaire # see Open CASCADE = > $ who, The Open CASCADE Object Libraries are. Oct 24, · Rick Astley - Never Gonna Give You Up ( Official Video) - Listen On Spotify: Learn more about the brand new album ' Beautiful.
Shop for Vinyl CDs more from Porcupine Tree at the Discogs Marketplace. Explore releases and tracks from Porcupine Tree at Discogs.
Porcupine Tree were an English rock band formed by musician Steven Wilson in 1987. The band began essentially as a solo project for Wilson, who created all of the band' s music. By late 1993, however, he wanted to work in a band environment, bringing on frequent collaborators Richard Barbieri as keyboardist, Colin Edwin as bassist, and Chris Maitland as drummer to form the first permanent lineup.


text:
Home // Award Winners // Lifetime Achievement Award
The Charles Hatchett Lifetime Achievement Award
The Charles Hatchett Special Award for Lifetime Achievement is an occasional Award, unlike the Charles Hatchett Award which is presented annually. The Award is based on an overall contribution to niobium metallurgy including publications, materials, product and process developments, and outstanding involvement over a period of time.
This Award is made with the agreement of the Institute of Materials, Minerals and Mining (IOM3).
Dr. Malcolm Gray
The Special Award was first made in 2008 as part of the 30th Anniversary of the Charles Hatchett Award. The first recipient was Dr. Malcolm Gray, and the citation read:
"...for his outstanding contribution over several decades of research, development, alloy design, processing, fabrication and applications of niobium microalloyed steel".
Dr. Malcolm Gray receives the first Lifetime Achievement Award from Barry Lye (President of IOM3) in 2008.
Find out more about Malcolm and his work by visiting the following website pages:
Charles Hatchett Award 2013 Seminar, presentation on Low Carbon HTP Structural Steel: A Paradigm Shift;
Charles Hatchett Award 2014 Seminar, presentation on HTP Process for Linepipe and Structural Steels;
Discovering the Importance of Niobium, Microalloying: A Lifetime of Learning video.
Dr. Phil Kirkwood
This Special Award was made in 2016 and the citation read:
"...for his outstanding development of Ti-B-Mo welding wires in the early 1970s and, since then, significant contributions to our understanding of the weldability of niobium-bearing steels and technical consistency in a broad range of complex metallurgical subjects".
Dr. Phil Kirkwood receives his Lifetime Achievement Award from Mike Hicks, President of IOM3 in 2016.
Find out more about Phil and his work below:
Dr Phil Kirkwood Bibliography.


text:
\section{Introduction}
Dynamical latent variable modeling has been widely used in characterizing multivariate time-series data\citep{molenaar1985dynamic}, including neural recordings\cite{truccolo2005point,brown1998statistical,eden2004Dynamic,fard2020analysis,yousefi2019state}. For time-series data analysis, two widely used modeling solutions are, a) state-space models (SSM)\cite{aoki2013state}, and b) neural networks including recurrent neural networks (RNN)\cite{chung2014empirical,hochreiter1997long} and deep neural networks (DNNs)\cite{rezaei2018comparison,zhang2018survey,van2016wavenet,tenney2019bert,collobert2008unified}. These solutions have been successfully applied in many applications; however, each solution is built upon prior assumptions that make their applications in certain time-series data challenging\cite{wan2001dual}.\\
In SSMs, we build a) a state process, that characterizes how the latent process(es) evolve over time, and b) an observation process, that represents the conditional distribution of observed signal given the state process. SSMs like any generative model require proper probabilistic models to characterize how the state process evolves over time, and how the observed data is generated given the state process. Building these models is challenging for high-dimensional observation processes, except we build a simplified observation model that leads to a biased or inaccurate inference. Furthermore, the assumption of conditional independence of the current observation from its previous time points given the state process might not hold in many time-series data. The recursive filter solution in SMMs considers this assumption; otherwise, the inference solution in SMMs becomes computationally intractable\cite{durbin2012time}.\\
DNNs and RNNs are deemed to address most of the challenges present in SMMs; however, these models are not designed to incorporate information at different timescales carried by the state and observation processes. This will limit applications of discriminative processes, like DNN and RNNs, in the inference of complex and intertwined dynamics. A critical advantage of SMMs is the way the state and observation processes at different timescales are combined in the characterization of data and inference step. Critical advantages of DNNs and RNNs are their precision power and scalability.\\
A new variant of SMM that has been recently introduced, called the direct discriminative decoder (DDD) model suggests a modeling pipeline to merge the advantages of SMM and DNN in a unified modeling framework \cite{Rezaei_neco_a_01491}. If the promises of DDD hold, we aim to expand its utilities in decoding complex and high-dimensional time-series data by embedding DNN and RNN in DDD.
The DDD then consists of a state transition process, as per SSMs, and a discriminative process in which the conditional distribution of the state, $\boldsymbol{x}_k$, is defined as a function of the current current observation, $\boldsymbol{s}_k$, and a subset of observations from previous time points $\boldsymbol{h}_k$. The filter solution in the DDD model is defined by
\begin{equation}
	\label{Eq-DDD}
	p(\boldsymbol{x}_{k}| \boldsymbol{s}_{1,...,k}) \propto
	\frac{p(\boldsymbol{x}_{k}|\boldsymbol{h}_{k},\boldsymbol{s}_{k})}{ p(\boldsymbol{x}_{k}|\boldsymbol{h}_{k})}
	{\int p(\boldsymbol{x}_{k}|\boldsymbol{x}_{k-1}) p(\boldsymbol{x}_{k-1}|\boldsymbol{s}_{1,...,k-1}) d\boldsymbol{x}_{k-1}}
\end{equation}
Where the nominator of the first term is called the prediction process and the denominator, using Chapman-Kolmogorov equation\cite{karush1961chapman} to expand $p(\boldsymbol{x}_{k}|\boldsymbol{h}_{k})=\int p(\boldsymbol{x}_{k}|\boldsymbol{x}_{k-1}) p(\boldsymbol{x}_{k-1}|\boldsymbol{h}_{k-1},\boldsymbol{s}_{k-1}) d\boldsymbol{x}_{k-1}$, called one-step prediction density by the prediction process from last time-step.  Here $p(\boldsymbol{x}_{k}|\boldsymbol{x}_{k-1})$ represents the state transition process and $\boldsymbol{h}_{k}=\{\boldsymbol{h}_{k-1},\boldsymbol{s}_{k-1}\}$. The second term in the right side of equation \ref{Eq-DDD} is the one-step prediction density by the posterior function from the last time-step defined by the Chapman-Kolmogorov equation. The DDD incorporates the advantages of SMM, a structure to incorporate information from slow timescales that present signals, and machine learning (ML) approaches to build expressive models using DNN and RNNs. \cite{Rezaei_neco_a_01491} and \cite{burkhart2020discriminative} showed the DDD performance precedes state-space and RNN models in both simulation and real data. However, DDD utility and learning were only demonstrated using simple discriminative models like the generalized linear model (GLM). The proposed learning solution for the DDD model is restrictive, given the denominator term contribution in parameter estimation is being ignored.
In this research, we propose two learning solutions for the DDD model including those with a DNN as its prediction process – we call a DDD with a DNN prediction process, D4. For the learning step, we derive a tight lower bound for the full log-likelihood of the state and observation and derive how DNN elements model including DNN parameters and history term can be learned. We demonstrate D4's performance in both simulation and real data and compare its result with DNN and SSM models. For the real-world application, we use the D4 model to decode 2-D trajectories of a rat moving through a maze from the ensemble spiking activity of its hippocampus place cells, 62 cells\cite{rezaei2021real}.
The DDD and D4 can be applied to a broader class of time-series analysis where the connection between high-dimensional observation and the underlying process is complex and is not known firsthand. These models can scale efficiently to high-dimensional data and they can be applied to many modalities of observed data including continuous, discrete, and or a mixture of them.

	\section{Methods}
	 Let's assume we have ${\boldsymbol{s}}_{{k}}\in \mathbb{R}^N$ for $k=1\ to\ K$, from a time series data. We also assume there is a latent process, ${x}_{k}$ for every time index $k$. The state transition process at time index $k$ is defined by
	\begin{equation}
	\label{eq_state_tran}
	\boldsymbol{x}_{k}|\boldsymbol{x}_{k-1} \sim g(\boldsymbol{x}_{k-1};\boldsymbol{\omega} )
	\end{equation}
	where $\boldsymbol{\omega}$ is the set of free parameters of the state equation.
    Here we expect that the observation at each time index is high-dimensional and possibly non-linear function of the latent state. The prediction process of the D4 is defined by
	\begin{equation}
	\label{eq_prediction_process}
	\boldsymbol{x}_{k}|\boldsymbol{s}_{k},\boldsymbol{h}_{k} \sim f(\boldsymbol{s}_{k},\boldsymbol{h}_{k};\boldsymbol{\Omega} )
	\end{equation}
	where  $\boldsymbol{h}_{k}$ is the history term at time index $k$ comprised of the observation from previous time points, and $\boldsymbol{\Omega}$ is the model free parameters. The inference in the D4 is defined by equation\ref{Eq-DDD}. Our learning objective is to find a set of parameters that maximizes the likelihood over the joint distributions of state variables and observations, $\{\boldsymbol{x}_{k},\boldsymbol{s}_{k}\}_{k=1}^{K}$.
	\begin{figure}[th]
		\centering
	\includegraphics[width=.6 \textwidth]{Figure1.png}
    \label{fig1}
    \caption{\centering Graphical representation of the D4 model. A) SSM , B)D4 models. $\boldsymbol{g}_{\Omega}(.)$ is the prediction process, possibly, a DNN, parameterized by $\Omega$. Further explanations of the DDD graph can be found in \cite{Rezaei_neco_a_01491} and $l$ is the maximum length of the history term.
 }\end{figure}
	\subsection{Parameter estimation in D4 model}
	Given having latent variables in the model, we use EM algorithm \cite{dempster1977maximum}. The expected value of log-likelihood function, $\boldsymbol{Q}$, is defined by
	$\boldsymbol{Q}$ as
	\begin{equation}
	\label{eq_lower_def}
	\boldsymbol{Q}(\boldsymbol{\theta}|\boldsymbol{\theta}^{(r)})={\mathbb{E}}_{{\boldsymbol{x}}_{{0:K}}|{\boldsymbol{s}}_{{1:K}};\boldsymbol{\theta}^{(r)}}
	[\log (p({\boldsymbol{x}}_{\boldsymbol{0}};\boldsymbol{\omega}_0) \prod^K_{k=1}p({\boldsymbol{x}}_{{k}}|{\boldsymbol{x}}_{{k-1}};\boldsymbol{\omega})
	\prod^K_{k=1}
	\frac{p({\boldsymbol{x}}_{{k}}|{\boldsymbol{s}}_{{k}},{\boldsymbol{h}}_{{k}};\boldsymbol{\Omega})}{ p({\boldsymbol{x}}_{{k}}|{\boldsymbol{h}}_{{k}};\boldsymbol{\Omega})
	})]
	\end{equation}
	where $\boldsymbol{\theta}^r = \{ \boldsymbol{\Omega}^r,\boldsymbol{\omega}^r \}$ represents model parameters estimated by maximizing $\boldsymbol{Q}$ from the previous iteration of EM algorithm, M-step\cite{Rezaei_neco_a_01491}. For the simplicity of notation, we use $\mathbb{E}_K$ for ${\mathbb{E}}_{{\boldsymbol{x}}_{{0:K}}|{\boldsymbol{s}}_{{1:K}};\boldsymbol{\theta}^{(r)}}$ in the rest of the paper.
	We can expand the $\boldsymbol{Q}(\boldsymbol{\theta}|\boldsymbol{\theta}^{(r)})$ function, which gives
	\begin{equation*}
	    \boldsymbol{Q}(\boldsymbol{\theta}|\boldsymbol{\theta}^{(r)})=\mathbb{E}_K
	[
	\log \; p({\boldsymbol{x}}_{{0}};\boldsymbol{\omega}_0) +
	\end{equation*}
	\begin{equation}
	\label{eq_Qvale_1}
	 \sum_{k=1}^{K} \log\; p({\boldsymbol{x}}_{{k}}|{\boldsymbol{x}}_{{k-1}};\boldsymbol{\omega}) +
	\sum_{k=1}^{K} \log \; p({\boldsymbol{x}}_{{k}}|{\boldsymbol{s}}_{{k}},{\boldsymbol{h}}_{{k}};\boldsymbol{\Omega})] -{\mathbb{E}}_K[
	\sum_{k=1}^{K} \log \; p({\boldsymbol{x}}_{{k}}|{\boldsymbol{h}}_{{k}};\boldsymbol{\Omega})
	]\end{equation}
	For the cases where the state process is linear with additive Gaussian noise and the prediction process is a multi-variate normal, we can find a closed-form solution for different terms of the $\boldsymbol{Q}$ function. When the noise characteristics in the prediction process is defined as a function of the observation or the state transition is nonlinear, it will be hard to find a closed-form solution for the $\boldsymbol{Q}$ function. It is possible to use sampling techniques to calculate the expectation for the first term; however, finding the expectation for the last term is challenging given it involves a second integration over the state. Instead, we show that we can find an approximation for the last term which turns into a lower bound for the $\boldsymbol{Q}$. We can write the last term in $\boldsymbol{Q}$ as
	\begin{equation*}
	  {\mathbb{E}}_K[
	\sum_{k=1}^{K} \log \; p({\boldsymbol{x}}_{{k}}|{\boldsymbol{h}}_{{k}};\boldsymbol{\Omega})
	]= \sum_{k=1}^{K} \int  p(\boldsymbol{x}_{k}|\boldsymbol{s}_{1:K},\boldsymbol{\theta}^{(r)})\log p({\boldsymbol{x}}_{{k}}|{\boldsymbol{h}}_{{k}};\boldsymbol{\Omega})d\boldsymbol{x}_{k}=
	\end{equation*}
	\begin{equation*}
	    =\sum_{k=1}^{K}\int  p(\boldsymbol{x}_{k}|\boldsymbol{s}_{1:K},\boldsymbol{\theta}^{(r)})\log \frac{p({\boldsymbol{x}}_{{k}}|{\boldsymbol{h}}_{{k}};\boldsymbol{\Omega}) p(\boldsymbol{x}_{k}|\boldsymbol{s}_{1:K},\boldsymbol{\theta}^{(r)})}{p(\boldsymbol{x}_{k}|\boldsymbol{s}_{1:K},\boldsymbol{\theta}^{(r)})} d\boldsymbol{x}_{k}
	\end{equation*}
	\begin{equation}
	  = \sum_{k=1}^{K} \int  p(\boldsymbol{x}_{k}|\boldsymbol{s}_{1:K},\boldsymbol{\theta}^{(r)})\log \frac{p({\boldsymbol{x}}_{{k}}|{\boldsymbol{h}}_{{k}};\boldsymbol{\Omega}) }{p(\boldsymbol{x}_{k}|\boldsymbol{s}_{1:K},\boldsymbol{\theta}^{(r)})} +\int p(\boldsymbol{x}_{k}|\boldsymbol{s}_{1:K},\boldsymbol{\theta}^{(r)}) \log p(\boldsymbol{x}_{k}|\boldsymbol{s}_{1:K},\boldsymbol{\theta}^{(r)}) d\boldsymbol{x}_{k}
	\end{equation}
	where the first term is the negative of $\mathbb{KL}$ divergence between the $p(\boldsymbol{x}_{k}|\boldsymbol{s}_{1:K},\boldsymbol{\theta}^{(r)})$ and $p({\boldsymbol{x}}_{{k}}|{\boldsymbol{h}}_{{k}};\boldsymbol{\Omega})$, and the second term is the negative of the posterior distribution entropy ${\boldsymbol{x}}_{{k}}$ given the whole observations parameters estimated from the previous iteration of the EM. Now, we can rewrite $\boldsymbol{Q}$ as
	\begin{equation*}
	\boldsymbol{Q}(\boldsymbol{\theta}|\boldsymbol{\theta}^{(r)})=\mathbb{E}_K
	[
	\log \; p({\boldsymbol{x}}_{{0}};\boldsymbol{\omega}_0) + \sum_{k=1}^{K} \log\; p({\boldsymbol{x}}_{{k}}|{\boldsymbol{x}}_{{k-1}};\boldsymbol{\omega}) +
	\end{equation*}
	\begin{equation}
	\label{eq_Q_d}
	\sum_{k=1}^{K} \log \; p({\boldsymbol{x}}_{{k}}|{\boldsymbol{s}}_{{k}},{\boldsymbol{h}}_{{k}};\boldsymbol{\Omega})]+
	    \sum_{k=1}^{K} \mathbb{KL}(p(\boldsymbol{x}_{k}|\boldsymbol{s}_{1:K},\boldsymbol{\theta}^{(r)}) \parallel p({\boldsymbol{x}}_{{k}}|{\boldsymbol{h}}_{{k}};\boldsymbol{\Omega}))+\mathbb{H}(p(\boldsymbol{x}_{k}|\boldsymbol{s}_{1:K},\boldsymbol{\theta}^{(r)}))
	\end{equation}
	The last two terms in equation\ref{eq_Q_d} are positive; thus, a lower bound for $\boldsymbol{Q}$ is the expectation over the state-transition and prediction process likelihood functions without taking the $\mathbb{KL}$ and $\mathbb{H}$ into account. This bound is unacceptable given it pushes the model parameter estimation in a favor of a prediction process with a long history term and possibly without incorporating the current observation, e.g non-informative about $\boldsymbol{x}_k$. If this is the case, the likelihood function, the first term in equation \ref{Eq-DDD}, becomes non-informative. On the other hand, the $\mathbb{KL}$ reaches its maximum when the prediction process is only a function of the current observation, e.g.  $p({\boldsymbol{x}}_{{k}}|{\boldsymbol{h}}_{{k}};\boldsymbol{\Omega})$ almost flat over the possible values of ${\boldsymbol{x}}_{{k}}$. This dynamics of $\mathbb{KL}$ suggests that there is an optimal history length that not only contributes to a better prediction of the state but also pushes $\boldsymbol{Q}$ to a higher value.
        We can use a greedy solution to incorporate both $\mathbb{KL}$ and $\mathbb{H}$ terms in the parameter estimation. Algorithm 1 presents the greedy optimization. The idea is to build D4 models with different history terms, whilst the parameter optimization per each D4 model is done without considering $\mathbb{KL}$ and $\mathbb{H}$ terms in \ref{eq_Q_d}. The D4 with optimal history term is the model with the highest $\boldsymbol{Q}$ values when the $\mathbb{KL}$ and $\mathbb{H}$ terms are added. Note that, we can calculate $\mathbb{KL}$ and $\mathbb{H}$ numerically or using sampling techniques when the model parameters are known. The downside of this greedy solution is that it makes the optimization computationally exhaustive. We discussed that the $\mathbb{KL}$ divergence values sweep as that length of history term changes, where its maximum happens when the prediction process is not a function of history term. When the prediction process is not a function of history term; the $\mathbb{KL}$ divergence can be approximated by negative of $\mathbb{H}$ plus a constant term where the constant term represent negative of log of intensity of the prediction process over possible values of the state. The sum of $\mathbb{KL}$ divergence plus the entropy will be a positive number that we want to maximize, therefore we want to maximize log of prediction process intensity. Note that the entropy term is only a function of the previous estimation of the model parameter; however, it provides a baseline for the  $\mathbb{KL}$ divergence. We assume the sum of the $\mathbb{KL}$ plus $\mathbb{H}$ should be larger than a pre-set positive threshold. With this assumption, we can approach our parameter optimization of $\boldsymbol{Q}$ from a regularization perspective. We define the regularization term by $(\mathbb{KL}(p(\boldsymbol{x}_{k}|\boldsymbol{s}_{1:K},\boldsymbol{\theta}^{(r)}) \parallel p({\boldsymbol{x}}_{{k}}|{\boldsymbol{h}}_{{k}};\boldsymbol{\Omega})) +\mathbb{H}(p(\boldsymbol{x}_{k}|\boldsymbol{s}_{1:K},\boldsymbol{\theta}^{(r)}))^2$, which can incorporate history length identification as part of the model training. Therefore, we can define the lower bound for the objective function as

\begin{minipage}{0.45\textwidth}
\begin{algorithm}[H]
\centering
\caption{Greedy optimization of D4 model parameters}
	\begin{algorithmic}[|1|]
	\small
	\label{alg_1}
        \State Set $\boldsymbol{h'}_k \xleftarrow{} \varnothing, l=0, \boldsymbol{Q}^{l} \xleftarrow{} 0$
		\Do
		\State $l=l+1$
		\State $\boldsymbol{Q}^{max}\leftarrow{} \boldsymbol{Q}^{l}, \boldsymbol{h}_{k}\xleftarrow{}\{\boldsymbol{s}_{k-l},\boldsymbol{h}_{k} \}$
		\State Initialize model parameters, $\mathbb{E}_K[.] \leftarrow{} 0$
		\Do
		\State $ \mathbb{E}^{max}_K[.] \leftarrow{} \mathbb{E}_K[.]$
		\State Run filter solution by equation \ref{Eq-DDD}
		\State Run filter Smoother by equation \ref{Eq-SmootherDDD}
		\State Draw sample trajectories for $x_k$ by equation \ref{Eq_SmTr}
        \State Find   ${\boldsymbol{\theta}}$ that maximizes $\mathbb{E}_K[.]$ term of the equation\ref{eq_Q_d}
        \doWhile{$ \mathbb{E}_K[.]>\mathbb{E}^{max}_K[.]$}
		\State $\boldsymbol{Q}^{l} \leftarrow{} \mathbb{E}^{max}_K[.] + \sum_{k=1}^{K} \mathbb{KL}(.)+\mathbb{H}(.) $
		\doWhile{$\boldsymbol{Q}^{l}>\boldsymbol{Q}^{max}$}
		\State Return the model associated with $\boldsymbol{Q}^{max}$
	\end{algorithmic}
\end{algorithm}
\end{minipage}
\begin{minipage}{0.45\textwidth}
	     \begin{algorithm}[H]
	\caption{Regularized optimization of D4 model parameters}
	\begin{algorithmic}[1]
	\small
	\label{alg_2}
        \State Set  $l>>1, \boldsymbol{h}_{k}\xleftarrow{}\{\boldsymbol{s}_{k-l:k-1} \}, \lambda\geq0, \boldsymbol{Q}^{l}\leftarrow{} 0$
		\State Initialize model parameters

			\Do
			\State $\boldsymbol{Q}^{max}\leftarrow{}\boldsymbol{Q}^l$
				\State Run filter solution by equation \ref{Eq-DDD}
		    \State Run filter Smoother by equation\ref{Eq-SmootherDDD},
		    \State Draw sample trajectories for $x_k$ by equation \ref{Eq_SmTr}
                 \State Find   $\boldsymbol{\theta}$ that maximizes
                  the $\boldsymbol{Q}^l$ in equation\ref{eq-objective-final}
			\doWhile{{$\boldsymbol{Q}^{l}>\boldsymbol{Q}^{max}$}}
			\State Return the model associated with $\boldsymbol{Q}^{max}$

	\end{algorithmic}
\end{algorithm}
\end{minipage}
	\begin{equation*}
	\boldsymbol{Q}(\boldsymbol{\theta}|\boldsymbol{\theta}^{(r)})\geq\mathbb{E}_K
	[
	\log \; p({\boldsymbol{x}}_{{0}};\boldsymbol{\omega}_0) + \sum_{k=1}^{K} \log\; p({\boldsymbol{x}}_{{k}}|{\boldsymbol{x}}_{{k-1}};\boldsymbol{\omega}) + \sum_{k=1}^{K} \log \; p({\boldsymbol{x}}_{{k}}|{\boldsymbol{s}}_{{k}},{\boldsymbol{h}}_{{k}};\boldsymbol{\Omega})]
	\end{equation*}
	\begin{equation}
	\label{eq-objective-final}
	-\lambda \sum_{k=1}^{K} (\mathbb{KL}(p(\boldsymbol{x}_{k}|\boldsymbol{s}_{1:K},\boldsymbol{\theta}^{(r)}) \parallel p({\boldsymbol{x}}_{{k}}|{\boldsymbol{h}}_{{k}};\boldsymbol{\Omega}))+\mathbb{H}(p(\boldsymbol{x}_{k}|\boldsymbol{s}_{1:K},\boldsymbol{\theta}^{(r)})))^2
	\end{equation}
	 where the multiplier $\lambda\geq 0$ is the regularization coefficient that puts implicit pressure to shrink history length. We can use a stochastic gradient ascent solution to find this lower bound; thus, we require to find gradients of different terms of $\boldsymbol{Q}$ respect to the model-free parameters.  We can calculate $\triangledown \boldsymbol{Q}(\boldsymbol{\theta}|\boldsymbol{\theta}^{(r)})_{\boldsymbol{\Omega}}$ by
	 \begin{equation*}
	          \triangledown \boldsymbol{Q}(\boldsymbol{\theta}|\boldsymbol{\theta}^{(r)})_{\boldsymbol{\Omega}} = \sum_{k=1}^{K}  \frac{\triangledown_{\boldsymbol{\Omega}} p({\boldsymbol{x}}_{{k}}|{\boldsymbol{s}}_{{k}},{\boldsymbol{h}}_{{k}};\boldsymbol{\Omega})}{p({\boldsymbol{x}}_{{k}}|{\boldsymbol{s}}_{{k}},{\boldsymbol{h}}_{{k}};\boldsymbol{\Omega})} - 2\lambda \sum_{k=1}^{K} \int p(\boldsymbol{x}_{k}|\boldsymbol{s}_{1:K},\boldsymbol{\theta}^{(r)})  ( \frac{\triangledown_{\boldsymbol{\Omega}}p({\boldsymbol{x}}_{{k}}|{\boldsymbol{h}}_{{k}};\boldsymbol{\Omega})}{p({\boldsymbol{x}}_{{k}}|{\boldsymbol{h}}_{{k}};\boldsymbol{\Omega})})d{\boldsymbol{x}_k}\times
	     \end{equation*}
	     \begin{equation}
	     \label{eq_lower_bound_grad}
	          (\mathbb{KL}(p(\boldsymbol{x}_{k}|\boldsymbol{s}_{1:K},\boldsymbol{\theta}^{(r)}) \parallel p({\boldsymbol{x}}_{{k}}|{\boldsymbol{h}}_{{k}};\boldsymbol{\Omega})) +\mathbb{H}(p(\boldsymbol{x}_{k}|\boldsymbol{s}_{1:K},\boldsymbol{\theta}^{(r)}))
	     \end{equation}
	     See more details in appendix A. Given equation\ref{eq_lower_bound_grad}, we can optimize the $\boldsymbol{\Omega}$ using a stochastic backpropagation similar to the ones proposed for variational autoencoder models \cite{kingma2013auto}. Algorithm 2 presents learning for D4 using stochastic backpropagation. So far, we assumed that $\boldsymbol{x}_{k}$ is unobserved, when $\boldsymbol{x}_{k}$ is known in the training dataset, the learning procedure follows the same steps where $\mathbb{E}_K$ is replaced by the observed trajectory of the state.
\subsection{Smoother estimation of the state, and sampling from the state posterior}
To be able to do model parameters optimization efficiently, we use a smoother on the posterior density\cite{sarkka2013bayesian}. The smoother update equation begins with the filter solution at time index $K$ at the end of the interval and proceeds backward from index $K$to index $1$. The smoother is defined by
	\begin{equation}
	\label{Eq-SmootherDDD}
	p(\boldsymbol{x}_{k}|\boldsymbol{s}_{1,...,K}) = p(\boldsymbol{x}_{k}|\boldsymbol{s}_{1,...,k}) \int \frac{p(\boldsymbol{x}_{k+1}|\boldsymbol{x}_{k})p(\boldsymbol{x}_{k+1}|\boldsymbol{s}_{1,...,K})}{p(\boldsymbol{x}_{k+1}|\boldsymbol{s}_{1,...,k})} d\boldsymbol{x}_{k+1}
	\end{equation}
   The smoother computes the posterior distribution of the state at time index $k$  given all of the observed data through time $K$. Now we are able to use a sampling method to optimize the prediction and state transition models' parameters using the lower bound described in equation\ref{eq-objective-final}.
	To draw samples from the state trajectory, we can use the conditional distribution of $\boldsymbol{s}_k$ given $\boldsymbol{x}_{k+1}$, and $\boldsymbol{s}_{1:K}$. This distribution can be computed by \cite{sarkka2013bayesian}
	\begin{equation}
	\label{Eq_SmTr}
	    p(\boldsymbol{x}_{k}|\boldsymbol{x}_{k+1},\boldsymbol{s}_{1:K}) = p(\boldsymbol{x}_{k+1}|\boldsymbol{x}_{k})\times
	    \frac{p(\boldsymbol{x}_{k}|\boldsymbol{s}_{1:k})}{p(\boldsymbol{x}_{k+1}|\boldsymbol{s}_{1:k})}
	\end{equation}
	To draw a trajectory of state posterior distribution from $k=1$ to $K$,
    we draw a sample from $p(\boldsymbol{x}_{K}|\boldsymbol{s}_{1:K})$, which is calculated by the equation \ref{Eq-SmootherDDD}. We then recursively draw samples from equation\ref{Eq_SmTr} for time steps $K-1$ to $1$. In cases, where the state is high-dimensional and computing the integrals in the filter and smoother equations using numerical methods becomes computationally expensive, we can use sequential MCMC methods, otherwise known as particle filters \cite{doucet2000sequential,doucet2009tutorial}.
\subsection{Simulation Dataset}
\label{syntheitc_data_sec}
We generate the simulation dataset by assuming the state of interest, $x_k$, is a one-dimensional random walk defined by
$P(x_k|x_{k-1})\sim N(ax_{k-1}+b,\sigma_x^2)$,
where $a$ is the state transition and $b$ is the bias coefficient for the random walk with a normal additive noise with a standard deviation of $\sigma_x$. Using the state, we then generate a 20-dimensional time-series signal with a conditional distribution defined by
\begin{equation}
    P(\boldsymbol{s}_k|x_k) \sim N(\boldsymbol{g}(x_k, x_{k-1},...,x_{k-l_m}),\boldsymbol{\Sigma}_s)
\end{equation}
where $\boldsymbol{g(.)}$ is a 20 dimensional vector of non-linear functions, like tanh, and cosine, with an argument defined by a subset of state process at the current and previous time points. The $l_m$ is the maximum number of data points used in creating $g(.)$ function. For each channel of data, we pick a $l$ uniformly from $0$ to $l_m$ randomly, which is used for the data generation. For example, in our simulation, the $g_1$ function corresponding to the first element of $\boldsymbol{g}$, is a $\tanh(.)$ with the argument to be $x_k+0.8x_{k-1}+0.6*x_{k-2}+0.4*x_{k-3}+0.2*x_{k-4}$. $\boldsymbol{\Sigma}_s$ is the stationary covariance  matrix with sizer $20\times20$. $\boldsymbol{\Sigma}_s$'s non-diagonal terms are non-zeros, implying observations across channels are correlated. In our simulation, we picked $a = .9$, $b = 0$, and $\sigma_x = 0.1$. We generate the data for 1000 data points. Figure\ref{fig:D4_synth_un_obs}.A shows a trajectory of the state and observed data. Our simulation data has a complex observation along with a low-dimensional state process; this setting was purposefully picked to demonstrate D4 prediction power and build a clear comparison across models. We also show the D4 model performance in neural data which can probe other aspects of the proposed framework and its training process.
\section{Results}
To demonstrate and compare DDD and D4 models capability in inferring the underlying state ($x_k$), we start with the simulation data described in section\ref{syntheitc_data_sec}. We assume the state process is defined by
\begin{equation}
    x_k = \alpha x_{k-1}+ \beta + w_k, \ \ w_k \sim N(0,{\sigma_x}^2)
\end{equation}
where $\sigma_x$ is the standard deviation of the state variable.
\begin{itemize}
    \item For DDD, we assume the prediction process is a linear model with an additive Gaussian noise, $p(x_k|\boldsymbol{s}_k, \boldsymbol{h}_k,\boldsymbol{\Omega})\sim N(\sum \boldsymbol{\Omega}_1 \odot \boldsymbol{s}_k + \boldsymbol{\Omega}_2 \odot \boldsymbol{h}_k, \sigma^2_s)$.
    Where $\boldsymbol{\Omega}_1$ and $\boldsymbol{\Omega}_2$ are the prediction process parameter sets associated to the current observation and the history; respectively.
    \item For D4, we assume the prediction process is a DNN model with additive Gaussian noise, $p(x_k|\boldsymbol{s}_k, \boldsymbol{h}_k,\boldsymbol{\Omega})\sim N( \mu_{\boldsymbol{\Omega}}( \boldsymbol{s}_k ,\boldsymbol{h}_k), \sigma_{\boldsymbol{\Omega}}( \boldsymbol{s}_k ,\boldsymbol{h}_k))$. Where $\mu_{\boldsymbol{\Omega}}( \boldsymbol{s}_k ,\boldsymbol{h}_k)$ and $\sigma_{\boldsymbol{\Omega}}( \boldsymbol{s}_k ,\boldsymbol{h}_k)$ are the mean and standard deviation of the Gaussian noise which both are nonlinear functions of the current, $\boldsymbol{s}_k$, and the history, $\boldsymbol{h}_k$.
\end{itemize}
Given the simulation data, we are trying to simultaneously update the state transition and prediction process parameters that maximizes the $\boldsymbol{Q}$ given $\boldsymbol{s}_k$.
It is worth mentioning that $\boldsymbol{s}_k$ is a vector of length 20, and $\boldsymbol{h}_k$ length 20 times the size of the history term. Figure\ref{fig:D4_synth_un_obs}.B shows the $\boldsymbol{Q}$ value as a function of history term for both DDD and D4 models. Using Algorithm 1, the optimal length of the history term is 5, whilst D4 has a higher $\boldsymbol{Q}$ value than the DDD model. Figure\ref{fig:D4_synth_un_obs}.C shows the $\boldsymbol{Q}$ values as function of $\lambda$ for both DDD and D4 models. A larger $\lambda$ penalizes the history term more which corresponds to a drop in the $\boldsymbol{Q}$ values as $\lambda$ grows. Optimal values of $\lambda$ for DDD and D4 models are 0.67 and 0.81; this is expected, as we expect larger penalization for the D4 given a higher number of free parameters. In Algorithm 2, we start with a fixed and reasonably large history term; the history term in both models was set to last 20-time points. We used mean -squared-error (MSE) and correlation coefficient (CC) to measure the accuracy of the D4 models in predicting state\cite{chai2014root}.  Figure\ref{fig:D4_synth_un_obs}.D shows these measures for both models while trained using Algorithms 1 and 2 with 2-fold cross-validation. The result suggests that the D4 model trained using Algorithm 2 gives the highest performance. This is not surprising; however, this performance has been maintained without observing an overfitting problem. Figure\ref{fig:D4_synth_un_obs}.E shows D4 model prediction in the test and training set (here we considered 50\% of data for training and the remaining 50\% for testing). The $\boldsymbol{Q}$ curve in Figure \ref{fig:D4_synth_un_obs}.B along with our prediction results suggest that DDDs benefit from the history term. Appendix B shows the prediction result for the DDD along with GRU-RNN and  SSM; among all these models, the D4 again maintains the highest performance.
\begin{figure}[th]
		\centering
		\includegraphics[width = .8\columnwidth]{Figure3.png}
		\caption{\centering The D4 modeling results for the simulated data. A) A sample of simulated data. B) $\boldsymbol{Q}$ function values as a function of history term using Algorithm 1. C) $\boldsymbol{Q}$ function values for different regularization terms, $\lambda$, using Algorithm 2. D) 2-fold cross-validated correlation coefficient (CC) and mean-squared error (MSE) for DDD and D4 models with the proposed algorithms. The D4 trained with Algorithm 2 attains the highest performance with 2-fold cross-validation. E) The decoding result (smoother) using the D4 in the training and test sets. }
		\label{fig:D4_synth_un_obs}
	\end{figure}
\subsection{D4 comparison with traditional time-series decoders}
We also compared the D4 model against the current time series decoder models, we compare its result with traditional SSMs\cite{truccolo2005point} and GRU-RNNs\cite{chung2014empirical}. We do that by considering similar prediction processes described in previous section and selecting proper functions for $\mu_{\boldsymbol{\Omega}}(\boldsymbol{s}_k,\boldsymbol{h}_k)$ and $\sigma_{\boldsymbol{\Omega}}(\boldsymbol{s}_k,\boldsymbol{h}_k)$ to represents each model. We try these three models on the simulation dataset (see appendix B for its results), described before, and on the position decoding problem from multi-electrode recorded neural activity of a rat while it forages a W-shaped maze for food\cite{rezaei2021real}. We measured the performance of the decoder models using mean squared error (MSE), mean absolute error (MAE) \cite{chai2014root}, and the 95\% highest posterior density region (HPD) metrics\cite{yao2009bayesian}. In this problem, we seek to decode the 2-D movement trajectory of a rat traversing through a W-shaped maze from the ensemble spiking activity of 62 hippocampal place cells. We define the state variable $\boldsymbol{c}_k=(x_k, y_k)$ where $x_k$ and $y_k$ represents the rat coordinates in the 2D spaces. The neural data were recorded from 62 place cells in the CA1 and CA2 regions of the hippocampus brain area of a rat.  Figure\ref{fig:D4_Maze_obs}.A shows the spiking activity of these 62 units, ${\boldsymbol{s}}_{{k}} \in \mathbb{R}^{62^2}$. For the state process, we use a 2-D random walk model. For the prediction process, we assume that the noise process follows a 2-dimensional normal distribution. As a result, the prediction process is a multi-variate regression model for $\boldsymbol{c}_k$s. For D4, we can assume the joint probability model can be written as the product of two predictor models,
\begin{equation}
    P(\boldsymbol{c}_k|\boldsymbol{s}_k,\boldsymbol{h}_k) = P(\boldsymbol{x}_k|\boldsymbol{s}_k,\boldsymbol{h}_k, y_k;\boldsymbol{\Omega}_x) \times P({y}_k|\boldsymbol{s}_k,\boldsymbol{h}_k;\boldsymbol{\Omega}_y)
\end{equation}
where each predictor model can be a DNN. Similar to the simulation data problem, we use a normal distribution where the mean and variance are a function of the current spiking activity of 62 cells and their spiking history. Along with D4, we also build SMM, explained in\cite{truccolo2005point,Rezaei_neco_a_01491} and GRU-RNN\cite{chung2014empirical} models. Figure\ref{fig:D4_Maze_obs} Shows their result. Among these models, D4 gives a  significantly better 95\% HPD compared to the SSM model. Note that this comparison is not possible for GRU-RNN, as its output is deterministic. The D4 model outperforms both SSM and GRU-RNN with regard to MSE, especially in the X-direction where the long-term temporal information is playing a significant role.
We think D4 performance precedes other models given it incorporates different temporal at moderate and slower timescales in its prediction of the rat position. In fact, this is aligned with the physiological of cells where their spiking activity is dependent on their previous spiking. This mechanism is not present in RNN, as in RNN, a single state that is governed by parameterized gates to reach a trade-off between long-term and short-term dynamics. As the result, the D4 model has a better performance in the X-direction. The other attribute of D4 is immunity to overfitting. As it can be seen in Figure\ref{fig:D4_Maze_obs}.I-J, the variation of D4 decoding results between training and test performance is significantly less than the GRU-RNN model. We think both the state process and regularized process in picking history terms play roles in decoding stability. This result suggests we can utilize the D4 in modeling problems with a smaller dataset, where GRU-RNN might face over-fitting issues.
 \begin{figure}[th]
		\centering
		\includegraphics[width = .95\columnwidth]{Figure4.png}
		\caption{ \centering Decoding of the rat trajectory from its hippocampus neural activity using D4, SSM, and GRU-RNN. A) an example of spiking activity of 62 cells recorded for 330 seconds while the rat is traversing in a W-shaped maze dataset. B) The rat observed position X\&Y-directions during the 330 seconds time. C) The D4 decoding result for Y-axis. D) The D4 decoding result for X-axis. E) The GRU-RNN decoding result for Y-axis. F) The GRU-RNN decoding result for X-axis. G) The SSM decoding result for Y-axis. H) The SSM decoding result for X-axis. We find the state transition parameter ($\sigma_x$) of the SSM empirically from the training set. I) The MAE results for D4, SSM, and GRU-RNN for the x- and y-axis. J) The MSE results for D4, SSM, and RNN for the x- and y-axis. K) The 95\% highest posterior density (HPD) interval for D4, SSM, and RNN for the x- and y-axis. The decoder models are trained using the first 80\% of data and the test is performed for the last 20\% data.
 }
		\label{fig:D4_Maze_obs}
	\end{figure}
\section{Discussion}
In this research, we introduced a new modeling solution to characterize multi-variate time-series data. We call the solution D4 model, which incorporates DNN into the recently proposed modeling solution called direct discriminative decoder, DDD. We revisited the inference solution using D4 and developed learning algorithms for the D4 model. We further demonstrated its solution in simulation and real datasets, where its performance preceded RNN, SSM, and DDD models.
The DDD is a variant of the SSM, in which the conditional distribution of the observation is replaced by a discriminative process. The D4 inherits attributes of DDD, while it provides higher expressive power for its prediction process by using DNNs. In SSMs, the inference process combines two sources of information at different time scales: a) fast, which incorporates the information carried at the current time observation, and b) slow, which incorporates the information on how the state evolves over a longer time scale. The DDD hypothesis is if the fast time scale is replaced by an intermediate time scale, we can better characterize time-series data . If this hypothesis holds, we can even push the DDD utility by using DNN as its prediction process. The results in both simulation and hippocampus place cells support this claim; we reach higher $Q$ values along with a better MSE and CC for D4 compared to DDD.
RNN and GRU have been successfully applied in characterizing time-series data where the observed signals reflect changes in an underlying latent process. RNN and GRU are mainly agnostic to the state process and search for the optimal history term through exploration or a gating mechanism. Instead, in the D4, the search from the history term is guided by the state process which is added to the model. As a result, in the worst case, we expect D4 performance to be on par with RNN and GRU. The simulation results show that D4 performance precedes RNNs; more importantly, D4 shows a less discrepancy in its performance for the test and training datasets. Thus, the D4 shows not only a higher prediction accuracy but also a more robust prediction.
In the discriminative processes, the focus is on the performance, not the probing underlying mechanism of how the data is generated. The D4 lives somewhere between generative and discriminative domains; thus, the D4 provides an etiological interpretation of our findings. This is important in many fields and in particular neuroscience. The Bayesian approach in formulating the D4 lets us add other observations to the model like those being used in SSMs. In other words, we can have both generative and discriminative observation processes. This is important in neuroscience as we have other bio-physiological correlated like reaction time (RT) with a simpler dynamic and can be embedded in the pipeline.
The problem presented here has relatively simple state processes, which are defined by random-walk models. The advantage of the random walk model is that they maintain continuity of the state, whilst they do not impose any further information like drift and complex dynamics across state variables. Practically, the D4 training is applicable to more complex state processes. Note that, the state process free parameters are present in the regularization term, and the choice of their free parameter will influence the history term of the prediction process. When the complexity of training and calculating the gradient of the regularization term grows, we can use the greedy training algorithm (Algorithm 1). It is worth noting that the inference will be the same as it is defined in equation\ref{Eq-DDD}.
We used a numerical approach to solve the learning step. To do this, we draw samples from the trajectory given the filter and a smoother solution. We calculate the filter and the smoother solution has numerically given the dimension of the state; we then draw samples from the trajectory which can be used in the estimation of the model parameters. When the dimension of the state grows, we can move to an MCMC solution to find the filter and smoother solution, which provide samples needed for the optimization.
Here, our focus was on motivating the D4 model and demonstrating its learning process. Our future research focuses on deploying the solution to a much larger hippocampus dataset which includes about 2000 channels of spiking activity. Furthermore, we are interested in studying hippocampus ripple events where there is no observation of the latent process, or projected movement directory, and we require to connect observed instances of the data to those which lack the state.
In the development of the D4, we took an ML approach to assess and compare model performance. We use MSE, CC, and $\boldsymbol{Q}$ value to show whether a model performance precedes others or not. We can approach the model comparison from a goodness-of-fit perspective to check how well the inferred states and model parameters represent the observed dataset. This will be another scope of our future research.
\section*{Broader Impact}
The state-of-art solution for decoding through high-dimensional time-series data and those with a complex dynamic are heavily dependent on discriminative models like DNN and RNNs. Despite a high expressive power, these models' performance decreases as we move from a static to dynamic inference even when the underlying latent process is low dimensional with reasonably predictable dynamics. The deterministic approach to the inference which is practiced in the majority of these models further limits their inference reliability and introduces high variance in the prediction. The remedy being practiced is relying on larger training sets along with growing the model complexity. Whilst this might work in problems that are mainly focused on prediction accuracy and access to a larger dataset is possible. When the scientific pursuit is on the inference and creating a larger dataset is not practical, we require solutions that can inherit attributes of the aforementioned solution whilst they are not data greedy. The D4 solution introduced in this research shows a decoding performance that precedes RNN and DNNs, with less variance in its prediction. The D4 model provides a principle solution to embed the state process information in DNN and RNN models; thus, its inference can have more ethological interpretation. The inference in the D4 is probabilistic rather than deterministic; this utility provides further clarity on the decoding step which is necessary for the scientific pursuit of what the data represents. The D4 application is not limited to neuroscience data, and it can be applied to a broad range of applications including NLP, and ASR. The D4 is scalable, and it can easily deal with the growth in the dimensional of the time-series data without changing the pipeline.
\section{Acknowledgments and Disclosure of Funding}
	This work was supported by Dr. Lankarany's NSERC Fund (RGPIN-2020-05868) and MITACS Accelerate (IT19240).

redpajama_set_name: RedPajamaArXiv


RS Sailing has introduced an open sailmaker format for its RS21 class racing.
And while RS admits historically the open sailmaker format often leads to large increases in the cost of purchasing sails it hopes to address this with a new concept.
And he explained that while open manufacture has been decided on, Dimension-Polyant will provide a proprietary laminate sail cloth to licensed RS21 sailmakers at a fixed cost and there will be a set maximum retail price to the end user.
In addition, RS will provide sailmakers with detailed class rules and build specification for all sails.
Customers will pay a royalty fee to the RS21 class through the sailmaker which will be used for RS21 class for class activity.
In addition, a class review will be conducted every three years to allow design changes to be discussed and the class to evolve.