text:
Tafsir Al-Jalalayn : The Great Commentaries of the Holy Qur'an Series Volume I
Reference: FV021
Up until now, the fundamental texts of the Islamic faith, apart from the notable exception of the Hadith of the Prophet, have remained out of reach of many Muslims and non-Muslims alike. This series aims to make widely available the leading exegetical works in translation for study and research in unabridged form, which are faithful to the letter and meaning of the Arabic.
Sahih Muslim With Full Commentary By Imam...
My First Quran With Pictures: Juz Amma Part 1
Tafsir al-Qurtubi – Vol. 5 Surat an-Nisa' 23 – 176
Tajweed For young Children – A Translation of...
Up until now, the fundamental texts of the Islamic faith, apart from the notable exception of the Hadith of the Prophet, have remained out of reach of many Muslims and non-Muslims alike. Among the most important sources for understanding the Qur'an are the tafsir works, commentaries on the Qur'an, which help to properly explain and contextualise the Revelation. Tafsir al-Jalalayn, presented here in complete English translation for the first time ever, is the first work in The Great Commentaries on the Holy Qur'an series. The series aims to make widely available the leading exegetical works in translation for study and research in unabridged form, which are faithful to the letter and meaning of the Arabic.
Tafsir Al-Jalalayn is one of the most significant tafsirs, or commentaries, for the study of the Quran. Composed by two "Jalals"- Jalal al-Din al-Mahalli (d. 1459CE) and Jalal al-Din al-Suyuti (d. 1505CE), Tafsir Al-Jalalayn is generally regarded as one of the most easily accessible works of Quranic exegesis beacause of its simple style and one volume length. For the first time ever Tafsir al-Jalalayn is completely translated into an unabridged, highly accurate and readable annotated English translation by Feras Hamza.
This Qur'anic commentary is one of the most popular Tafasir in the Islamic world, perhaps even THE most popular Tafsir. Copies of it are available in almost every bookshop and library in the Arab and Muslim world. It is by far the shortest and easiest to understand. Consequently, it is invariably read as an introduction to Classical Tafsirs by millions of students and adults who never go further into the subject. It is thus an immensely successful and influential work not just as the classic introduction to Tafsir, but also as the standard reference work for the language of Qur'an.
Al-Jalalayn's two author's deliberately summarized, streamlined or simplified the material in order to stay focused on their one overriding aim: to make the literal meaning of the Holy Qur'an completely intelligible in the simplest possible way.
H.R.H. Prince Ghazi Bin Muhammad is Personal Envoy and Special Advisor to H.M. King Abdullah II, and the Chairman of the Board of Trustees of The Royal Aal al-Bayt Institute for Islamic Thought.
About Authors:
Jalalu'd-din Muhammad ibn Ahmad al-Mahalli (791-864/1389-1459) of Cairo was a versatile, scholar who excelled in jurisprudence, theology, grammar, rhetoric, and Qur'anic commentary. He was known for his scrupulousness, fear of Allah, and fearlessness in upholding the truth. Al-Mahalli was offered the highest judicial positions but refused them. He taught jurisprudence in the Mu'ayyadiyya and Barquqiyya madrasas. Abstinent and ascetic, he lived on what he earned by trade. His most famous work is his Qur'anic commentary, Tafsir al-Jalalayn, which he began halfway through the text with Surat al-Kahf, ending with an-Nas and al-Fatiha.. Although he died before he could start the other half, the work was completed by his student, Jalalu'd-Din as-Suyuti. Al-Mahalli's other books include commentaries on Jam' al-Jawami', al-Burda, al-Manahij fi'l-fiqh, al-Waraqat fi'l-usul, Kitab al Jihad.
Imam Jalal al-Din al-Misri al-Suyuti al-Shafi`i al-Ash`ari, also known as Ibn al-Asyuti (849-911AH /1445-1505), the mujtahid imam and renewer of the tenth Islamic century, foremost Hadith master, encyclopaedist, historian, and biographer and probably one of the most prolific of all Islamic writers. There are an enormous number of his essays and treatises preserved today. A number of his writings concerned scientific topics or issues related to natural science and food and regimen, amongst other things From Asyut in Egypt, he was among the most renowned and prolific Muslim scholars of all time. He wrote more than 300 books, covering every aspect of the Islamic sciences. He memorized the Qur'an at the age of 'eight, and then went on to study with more than 150 scholars. He traveled extensively in his quest for knowledge — to Damascus, Hijaz, Yemen, India, Morocco, and the lands south of Morocco, as well as in Egypt.
Al-Suyuti devoted his life to learning, teaching and writing. He was noble, abstinent and self-sufficient, distancing himself from people of rank and power and living on what he earned by teaching. Major writings of al-Suyuti that remain widely used today include al-Itqan, on the Qur'anic sciences; and Tafsir al-Jalalayn, which he completed when only 22.
About Translator:
Dr. Feras Hamza received his M. Phil and D. Phil in Islamic history from the University of Oxford. He has worked as a research assistant in Islamic history at the Institute for Advanced Study in Princeton and is currently a Research Associate with the Qur'anic Studies Project at the Institute of Ismaili Studies in London. He is a specialist on early Islam and teaches courses in Islamic History, Quran and Tafsir. Dr. Hamza is presently at work on a translation of Al-Baydawi's Tafsir and a monograph on the Birth of the Muslim Afterlife.
Author Jalal al-Din al-Mahalli, Jalal al-Din al-Suyuti
Translator Dr. Feras Hamza
Publisher Fons Vitae
Height 1.7 in
Width 8.0 in
Tafsir Al-Jalalayn : The Great...


Handmade Sterling Silver Bloodstone and Clear Quartz Necklace.
Bloodstone is a courage gemstone, it restores ones determination, and it is said to assist in regaining ones power.
The product is 41 cms long, but it is possible for the product to be made for your specific requirements. If this is what you require please enter your size in the special requirements box below.


Freedom the power or right to act, speak, or think as one wants. Or the state of not being imprisoned or enslaved.
In many parts of the world, freedom is something that is taken for granted. Take for example, the freedom to choose any religion we want (or no religion at all). Or to be in a relationship with the person we love, to travel…the list goes on.
Unfortunately, there are many places in the world where these freedoms are not available to most people. There are still people in the world who are forced to live empty, unfulfilling lives so the rulers of their countries don't have them thrown in jail or worse.
If you don't know what it means to be afraid to voice your opinion, then you can consider yourself very lucky to live in a place where liberty is a priority as well as a basic human right.
And like all good things, liberty should be celebrated, and that's what World Freedom Day is all about.


@Aman_B I've resent the PM, even though I can see the original in my Sent Messages.
Got it @IceMan61. Will reply there.
I am still unable to log in to my account and to be bluntly honest I find this to be a ridiculous amount of time for the issue to be resolved.
It's been just under 2 weeks since I raised a technical issue with support.
It is a complete waste of time contacting optus via Facebook messenger as the staff are unhelpful and don't seem to read previous messages.
Based on this issue alone I will never recommend any optus product or service to anyone.
Simply put it has been a poor experience from day one.
Hi all, I still can't log in to my account.
Is the issue really that bad/complex that it takes 2 plus weeks to resolve?
Finally after calling back I can now log into My Account !!
Great to hear that this has been resolved, @Mudsy360. If you need a hand with anything else moving forward, please don't hesitate to reach out.


text:
Season Four is Here.
Welcome to The Detroit History Podcast. In our fourth season, we'll explore The Scene, which had an outsized impact on Motor City culture as the signature show on Detroit's only African-American owned television station; explore the construction of the Detroit-Windsor tunnel 90 years ago; study the life of a 19th century Michigan lumber baron as we look for ghosts in The Whitney restaurant. And we'll tell you about the Nike Missiles that were stationed around metro Detroit at the height of the Cold War.
You can find us on Apple Podcasts, Spotify, Stitcher, and our embedded audio player on this website. Tim Kiska hosts the program, with new episodes being released each Sunday evening. Hope you enjoy.
Season 4 – Finale: Jeff Montgomery, Detroit's Fierce LGBTQ+ Rights Activist
dethistadmin 0 Comments
Jeff Montgomery was a born activist who played an important role in saving Orchestra Hall. When a hate crime brought tragedy to his personal life, he channeled his talent and drive to working on behalf of the LBGTQ+ community. His stellar career and sad...
https://detroithistorypodcast.com/podcast-player/1385/jeff-montgomery-detroits-fierce-lgbtq-rights-activist.mp3
Season 4 – Episode 9: When The Cold War Seemed Hot: Nike Missiles Around Metro Detroit, And A Nuclear Warhead On Belle Isle
Between 1955 and 1974, a nuclear war with the Soviet Union seemed like a possibility. We armed ourselves by placing Nike Missiles around many major cities across the U.S. — including 16 in and around metro Detroit. Six of them — including one on...
https://detroithistorypodcast.com/podcast-player/1382/nike-missiles-around-metro-detroit-and-a-nuclear-warhead-on-belle-isle.mp3
Download file | Play in new window | Recorded on December 5, 2021
Season 4 – Episode 8: Hammerin' Hank Greenberg, How a Jewish Kid from the Bronx became a Detroit Tiger Great
Hank Greenberg, who entered the Hall of Fame as one of the greatest hitters in the game's history, was the first Jewish star in team sports. He interrupted his baseball career to serve longer in World War II than any other major league player,...
https://detroithistorypodcast.com/podcast-player/1379/hammerin-hank-greenberg-how-a-jewish-kid-from-the-bronx-became-a-detroit-tiger-great.mp3
Download file | Play in new window | Recorded on November 22, 2021
Season 4 – Episode 7: The 1960s and General Motors: Consumerism Hits The Big Three
GM spied on a gadfly and got caught. It was the '60s, and it changed the auto industry forever. When consumer advocate Ralph Nader began hounding Detroit to produce safer cars, the world's largest corporation took affront and went snooping. Its chairman, James Roche,...
https://detroithistorypodcast.com/podcast-player/1376/season-4-episode-7-the-1960s-and-general-motors-consumerism-hits-the-big-three.mp3
Season 4 – Episode 6: No-No Boy and the Japanese-American Migration to Detroit
dethistadmin 29:19 0 Comments
Barely two months after the attack on Pearl Harbor, President Franklin Roosevelt issued Executive Order #9066. Some 120,000 Japanese Americans in this country's western states were ordered into internment camps. We report on the order, and the post-war period. When the camps were finally...
https://detroithistorypodcast.com/podcast-player/1372/season-4-episode-6-no-no-boy-and-the-japanese-american-migration-to-detroit.mp3
Season 4 – Episode 5: Vernors, The Nectar of the Gods
James Vernor invented his ginger ale in downtown Detroit just after the Civil War. More than 15 decades later, we're still fans. The Detroit History Podcast tells the story of this enduringly popular soft drink. You thought Vernor Highway in southwest Detroit was named...
https://detroithistorypodcast.com/podcast-player/1369/season-4-episode-5-vernors-the-nectar-of-the-gods.mp3
Stay updated with the show and connect with other listeners by following our social media feeds.
The hardworking Detroit History Podcast is a collaborative effort.
Tim Kiska
Tim Kiska is a professor at the University of Michigan–Dearborn. He worked in journalism at the Detroit Free Press (1970-1987), Detroit News (1987-2002) and WWJ-AM, the CBS-owned all-news radio station. He earned his PhD in history at Wayne State University in 2003, two years after joining the UM-D faculty.
Eric Kiska
Born in Detroit, MI, Eric Kiska is a musician, writer, and lover of the outdoors. Eric graduated with a Bachelors of Science in English Writing and a minor in Art & Design from Northern Michigan University in 2014. Shortly after he had a part-time column for the Motor City Muckraker focusing on music around Detroit. He has also done freelance video editing for Detroit Public TV's mivote.org, and helped raised $21,000 for the "BC Surgery Stories" kickstarter campaign by editing the kickstarter video and assisting in social media promotions. Eric is now a mail carrier in Sterling Heights, MI while continuing to pursue a career in writing.
Bob Koski
Bob Koski is a lifelong Detroiter who became warped at the age of eight after seeing the Beatles on Ed Sullivan in February, 1964. Through a sequence of classical piano lessons, electrical engineering, club deejaying, and eventually being a producer/song parody writer for "J.J. & The Morning Crew" on Detroit radio, he has now morphed into an audio engineer–digitally preserving archival recordings. His love of Beatles music still dominates his life, and he actually got to play one of the Abbey Road studio pianos used in many Beatles recordings.
Bill Kubota
Bill has produced, written, directed and lensed local, regional and national documentaries for broadcast since the 1980s. His national projects include co-producing the Dupont Columbia Journalism award-winning Beyond The Light Switch series for Detroit Public Television. He recently directed the PBS documentary The Ethanol Effect for Detroit Public Television. With his production company, KDN Films, he directed Lustron – The House America's Been Waiting For working with Ohio State University Public Television and the primetime national network broadcast of Most Honorable Son presented by PBS and NET Nebraska. His work has also appeared on PBS NewsHour, ABC News, CBS News and ESPN. He's worked as a producer, photographer, on-air reporter and news assignment manager for television stations in Lansing, Flint and Detroit. He's currently working on short films that look at the major issues facing Detroit for DPTV, One Detroit and DPTV's Great Lakes Now.
Kelley Kiska
Kelley Kiska is an art, design, pop culture, and information nerd from Grand Blanc, MI. She has been designing and creating visual solutions since 2014, and graduated with a Bachelors of Fine Arts in Graphic Communication from Northern Michigan University in 2016. She has created logos and identities for various industries including real estate and online gaming. Most of her days are spent in the digital sphere, but she is also passionate about working with her hands; diving into calligraphy and illustration. She is currently working as a graphic designer and web developer at a paid media shop outside of Ann Arbor.
Tom Ferguson
Ferguson joined the Detroit Free Press news desk at the end of 1967-'68's ninety-day newspaper strike, and spent two decades in various editorial jobs. He also worked at the Cleveland Plain Dealer, edited America's Largest Country Weekly, published a music magazine, edited a health-care newspaper, managed a short-run book publisher, and continues to play piano whenever he finds a bar and grill that will have him.
To receive ongoing updates and notifications on new episodes, please sign up to our Newsletter and we'll make sure to keep you in the loop!
Season 4 – Finale: Jeff Montgomery, Detroit's Fierce LGBTQ+ Rights Activist December 20, 2021
Season 4 – Episode 9: When The Cold War Seemed Hot: Nike Missiles Around Metro Detroit, And A Nuclear Warhead On Belle Isle December 5, 2021
Season 4 – Episode 8: Hammerin' Hank Greenberg, How a Jewish Kid from the Bronx became a Detroit Tiger Great November 22, 2021
Season 4 – Episode 7: The 1960s and General Motors: Consumerism Hits The Big Three November 15, 2021
© 2020. The Detroit History Podcast.

redpajama_set_name: RedPajamaCommonCrawl


Sheffield Burngreave Cemetery F2. C. 56.
Gezaincourt Communal Cemetery Extension I.D.2.
Husband of Annie Eliza Abbott of 8/3 Gibralter street.
Cite Bonjean Military Cemetery Armentieres IX. G. 58.
Husband of Annie Elizabeth Catherall (formerly Abdy) of 37 Fitzalan street Pitsmoor.
Son of Herbert and Margaret Abey of 32 Witney street Sheffield.
Boulogne Eastern Cemetery VIII. I. 196.
Son of Mrs Ada Abson husband of Edith Abson of 214 Dunlop street Sheffield.
Husband of Fanny Acock of 12 Court 5 House Fitzwilliam street Sheffield.


The article "Rethinking Behavior Genetics" targets members of the scientific community in that it seeks to highlight differences between the first 100 years of studies in behavior genetics and the current century of the studies. Additionally, the author tries to focus on the major problems facing the current studies in behavior genetics on one hand, and the recommended solutions to the problems on the other hand (Hamer 71-72). Accordingly, the author manages to effectively draw a distinction between the first 100 years of studies in behavior genetics and the present day studies on the same topic by noting that the former studies produced excellent results because the approaches followed were straightforward. During the first century, it is apparent that almost all studies were centered on finding the link between genes and various aspects of behavior and such aspects as personality, psychiatric disorders, cognitive styles, and temperament among others (Hamer 71). However, the author fails to review any successful study from this period to illustrate his view-point. Here, despite most studies documenting that different effects of heredity were central to variations in behavior, thoughts, and feelings, there is little evidence to show the direct link to date.
As a result, Hamer (71) notes that the second century of studies in behavior genetics is marked by efforts to research specific genes responsible for individual differences in behavior. Further, the author notes that the findings from such studies are less than satisfying considering that to date none of the studies published has produced conclusive results. This is mainly because many scientists are pursuing the same line of investigation as their fellow researchers in the past century whereby it was assumed that the relationship between genes and behavior is direct. However, Hamer (71) prefers a different line of investigation, which takes into account the effect of gene networks and the environment (parents, economics, siblings, and education) in the cognitive processes underlying different behaviors.
From the studies reviewed in the article, it is apparent that different factors related to the environment affect gene networks, which in turn contribute to different behaviors. For example, the serotonin transporter gene has been shown to influence the response of the amygdala region to various fearful stimuli through a DNA sequence, which is also associated with atypical levels of anxiety (Hamer 71). As a result, there is an evidence to suggest that behavior is a manifestation of various interconnected systems, which work together to produce a specific action. These systems have been narrowed down to gene networks, environmental conditions, and the cognitive processes. On the other hand, Hamer (72) notes that the second century of studies in behavior genetics is ailing from the lack of biological logic in designing research methods, which can produce satisfactory results. In as much as this is a contributing factor to the problems experienced in research today, it is not entirely to blame considering that there are other underlying factors such as limited resources, socio-medical politics, and lack of mechanisms to replicate animal studies using human subjects, which may hinder the ability of contemporary studies to go into conclusion.
Ultimately, Hamer (72) notes that the future of behavior genetics should be guided by the development of mechanisms, technologies, and techniques, which can be used to study the simultaneous activities of different genes implicated in a specific behavior. This is very true considering that the human brain is a complex network of circuits working around numerous genes to produce different behaviors. As a result, the development of techniques and equipment with the functionality of microarray analysis will shape the future of behavior genetics by enabling scientists to study various activities of gene networks in the production of specific behaviors. Overall, this article is very effective in highlighting various weaknesses of the on-going studies in behavior genetics besides enlightening members of the scientific community on the need to embrace technological advancements and biological logic in their present and future studies.


text:
Q: SQL Subquery Where Column = Column -10 SELECT

worksorders.id AS 'Works Order No',
worksorders.partid AS 'Part No',
allpartmaster.partdesc AS 'Description',
worksorders.quantity AS 'Qty',
wip.wqleft AS 'Qty Rem',
worksorders.duedate AS 'Due Date',
worksorders.lateststartdate AS 'Latest Start Date',
worksorders.kitstatus AS 'Kit Status',
wip.wopnumber AS 'Op No',
wip.wmachine AS 'Work Centre',
wip.wdesc AS 'WC Description',
wip.wstate as 'Op State',
(SELECT wstate FROM wip WHERE wip.wona = worksorders.id and wip.wopnumber = (wip.wopnumber - 10)) as 'Prev Op State'

FROM efacdb.dbo.allpartmaster allpartmaster, efacdb.dbo.wip wip, efacdb.dbo.worksorders worksorders

WHERE worksorders.id = wip.wona AND allpartmaster.partnum = worksorders.partid AND ((worksorders.status = 'ACTIVE') AND (wip.wstate = 'NS'))

AND wip.wmachine = 'POLISH'

ORDER BY worksorders.lateststartdate ASC

I'm having trouble with this query, I'm not sure if it's even possible. I'm returning operations which are numbered in steps of 10, I need to return the status of the previous operation. Here's the key line:
(SELECT wstate FROM wip WHERE wip.wona = worksorders.id and wip.wopnumber = (wip.wopnumber - 10)) as 'Prev Op State'

I want to take the wopnumber from the parent SELECT query, take 10 off it and return the wstate column of that row.
Is this possible?

A: Maybe this is what you want?
SELECT

        worksorders.id AS 'Works Order No',
        worksorders.partid AS 'Part No',
        allpartmaster.partdesc AS 'Description',
        worksorders.quantity AS 'Qty',
        wip.wqleft AS 'Qty Rem',
        worksorders.duedate AS 'Due Date',
        worksorders.lateststartdate AS 'Latest Start Date',
        worksorders.kitstatus AS 'Kit Status',
        wip.wopnumber AS 'Op No',
        wip.wmachine AS 'Work Centre',
        wip.wdesc AS 'WC Description',
        wip.wstate as 'Op State',
        app.wstate as 'Prev Op State'

FROM efacdb.dbo.allpartmaster allpartmaster, efacdb.dbo.wip wip, efacdb.dbo.worksorders worksorders
     cross apply (SELECT wstate
                  FROM wip wip1
                  WHERE wip1.wona = worksorders.id and wip1.wopnumber = wip.wopnumber - 10) app

WHERE worksorders.id = wip.wona AND allpartmaster.partnum = worksorders.partid AND ((worksorders.status = 'ACTIVE') AND (wip.wstate = 'NS'))

AND wip.wmachine = 'POLISH'

ORDER BY worksorders.lateststartdate ASC

A: Maybe something like this?

SELECT
worksorders.id AS [Works Order No],
worksorders.partid AS [Part No],
allpartmaster.partdesc AS [Description],
worksorders.quantity AS [Qty],
wip.wqleft AS [Qty Rem],
worksorders.duedate AS [Due Date],
worksorders.lateststartdate AS [Latest Start Date],
worksorders.kitstatus AS [Kit Status],
wip.wopnumber AS [Op No],
wip.wmachine AS [Work Centre],
wip.wdesc AS [WC Description],
wip.wstate as [Op State],
prevwip.wstate as [Prev Op State]
FROM efacdb.dbo.allpartmaster allpartmaster
JOIN efacdb.dbo.wip wip ON (allpartmaster.partnum = worksorders.partid AND wip.wstate = 'NS' AND wip.wmachine = 'POLISH')
JOIN efacdb.dbo.worksorders worksorders ON (worksorders.id = wip.wona and worksorders.status = 'ACTIVE')
LEFT JOIN efacdb.dbo.wip prevwip ON (prevwip.wona = wip.wona and prevwip.wopnumber = (wip.wopnumber - 10))
ORDER BY worksorders.lateststartdate ASC

Disclaimer: Only tested in notepad

redpajama_set_name: RedPajamaStackExchange


I've finished these without problems.
but i'm stuck in Setting up PlayFab title for Photon steps.
"Photon add-on page will allow to install Photon by specifying you Photon Realtime Application ID (1). Optionally, you may specify Chat Application Id. Once you install the add-on (3), Secret Key will be generated (2). Save it, as it will come in handy when configuring web-hooks:"
Thanks for calling that out - I'll get a bug opened on this. The secret is being generated for titles - I'll send you yours via email.


Our Cattail weekend retreat has ended. The projects worked on will be around for a good, long time. Quilt tops sewn, sun dress (and accessories planned) for a little granddaughter was stitched, photos scrapbooked (yes, actual paper copies, not digital ones!) and a new hobby was learned, paper piecing fabric to make mini works of art.
We did have dinner with lentil soup first…salmon spread, homemade deli rye buns, beets from Mom and Dad's garden, Carr Valley cheeses, sprouts and capers for greens. Yum!
Saturday lunch was pheasant wild rice soup, taco dip and sandwiches with tiramisu. Saturday dinner was chicken enchiladas, refried beans, rice and chocolate pudding layer cake which was frozen so it was like an ice cream treat. Sunday is leftover day plus bonus chocolate cake. Plus samples of Nessalla hibiscus cardamom ginger kombucha and home brew peach kombucha and moscato. All very delicious!
Pillow gifts…fabric bowls with truffles from Dawn, hearts themed potholder (too nice to use this way, it will be a table center piece for a bit), pillow box from me compliments of Paper Pumpkin.
My new experience for the weekend was Dawn teaching me how to paper piece quilt blocks. The teapot took about 6 hours but took breaks and had dinner in that time too. The teacup (not pictured) took about 90 minutes the next day including a sewing machine tune up with Pauline's help.
The scrapbooking girls..Jeanette, me and Tiffany. Tiffany zoomed home first thing this morning due to the snowy weather. She got home in good time but Jeanette was slowed by the weather.
We had a fun and productive weekend making memories in a country setting. Looking forward to next time when we just have the August heat to contend with!
Click here for a tour of the retreat house starting in the craft room. I filmed the video in Facebook so add a comment if you are able to view it. This is the first time I tried sharing a Facebook Live video to my blog.


text:
Lisa operating system source code
January 8th, 2018 8:52 AM
by Ken Gagne Filed under History;
Comments Off on Lisa operating system source code
Last week, I wrote about Robert Taylor and Charles Thacker, whose work at Xerox PARC inspired such Apple innovations as the graphic-user interface (GUI) and the mouse. Soon, we'll get to see under the hood of how Apple introduced those technologies with the Apple Lisa.
Just as the Computer History Museum did five years ago when it released the source code for Apple DOS, the CHM will now be distributing the source code for the Apple Lisa's operating system. Museum curator Al Kossow made the announcement on Google Groups, writing, "the sources to the OS and applications were recovered… and they are with Apple for review. After that's done, CHM will do an @CHM blog post about the historical significance of the software and the code that is cleared for release by Apple will be made available in 2018."
I'm curious where the source code was "recovered" from. Did the CHM collaborate with Apple to retrieve the code from an archaic floppy disk, much as Tony Diaz and Jason Scott helped Jordan Mechner recover the Prince of Persia source code? Or did some third party, perhaps a former Apple employee, bequeath the code to the CHM?
Regardless of the source, the importance of this release cannot be understated. Rhett Jones at Gizmodo reported, "Lisa was a cutting-edge machine and one of the first to offer consumers a GUI, mouse, and file system, but it was prohibitively expensive and didn't catch on." To see the origin of these features is to look back at the ancestors of computing staples that are still with us today.
Further, such releases are extremely rare, as Apple is known to be possessive of their intellectual property. In this case Apple has little incentive to make such a release, whether or not there is historical value or modern applications for the Lisa operating system.
Whatever the origin or motivation of both this release and that of Apple DOS before it, the precedents continue to be set, with many implications for the Apple II community. Who knows what other classic software we'll see released from Apple Inc. next?
(Hat tip to Christopher Baugh via Paul Wilson)
Why is Apple DOS source code release important?
by Ken Gagne Filed under History, Mainstream coverage;
Last week, the Computer History Museum released the source code to Apple DOS 3.1:
With thanks to Paul Laughton, in collaboration with Dr. Bruce Damer, founder and curator of the Digibarn Computer Museum, and with the permission of Apple Inc., [the Computer History Museum is] pleased to make available the 1978 source code of Apple II DOS for non-commercial use. This material is Copyright © 1978 Apple Inc., and may not be reproduced without permission from Apple.
This disk operating system has been previously decompiled and the notes published online, which you'd think would make the Computer History Museum's release unimportant. And yet it's been reported far and wide — not just the usual vintage computing sites, but Apple news sites and broader IT outlets as well. Why all the commotion over old news for a 30-year-old machine?
It's not the product, but the process, that's significant. "What is interesting is Apple's agreement to release it," said Dr. Steve Weyhrich of Apple II History. "They never release stuff like this to the world." Agreed David Schmidt: "That is the only thing that is interesting about this: Apple's actual permission to leak any kind of intellectual capital."
However, there is also some original material in this particular offering of the source code. Weyhrich continued: "The code that was released also has that advantage of being scanned from actual printouts of Paul Laughton's work in progress, with his comments on how the different parts of the system work… I've seen some of these [documents] before, but there are some that not even David Craig has gotten his hands on over the years. Historically, it's quite interesting, and the Museum has comments from Laughton on the process of the creation of Apple DOS that give details that I didn't get out of him when [I interviewed him — I guess] I didn't ask the right questions!"
Click past the jump for an index of sites that have reported the source code's release.


The machine can be used in each scientific and business purposes, but used primarily scientific functions. A D-Wave quantum processor learned to tell timber from rivers, fields, and roads. It helps to build our worldwide editorial crew, from war correspondents to investigative reporters, commentators to critics. The opposite, pursued by IonQ and several main tutorial labs, is to encode qubits in single ions held by electrical and magnetic fields in vacuum traps.
And if all the pieces goes to plan, the forty nine-qubit chip will make Google the primary to construct a quantum pc able to solving certain issues that are past the talents of ordinary computers. Now, a new contest for technological supremacy is brewing, and while the United States and Russia are definitely at play, China and even private enterprise have also joined the fray.
That may not seem to be an enormous deal, however the end result implies that researchers are a step closer to using such computers for sophisticated machine learning issues like sample recognition and laptop vision. The interior ministry said on its website about 1,000 computer systems had been infected but it surely had localised the virus.
The Seattle-based mostly tech large issued steering for individuals to protect their techniques, whereas taking the highly unusual step of reissuing security patches first made out there in March for Home windows XP and other older variations of its operating system. The right way to shield your Home windows computer from the Petya ransomware assault.
Info and Communication Expertise: Affects on U.S. College College students. Faculty college students' perceived attributes of Internet websites and online procuring. In drugs, this era introduced improvements corresponding to open-heart surgical procedure and later stem cell remedy together with new medicines and coverings. In terms of applied sciences, we mainly need to make things simple.
As some American corporations pull back from hiring programmers a world away, new tech jobs are cropping up across the nation. In the early 20th century," writes one researcher, walking up to one hundred km/day was not unusual." However these days are over, changed by trendy comforts.
Naturally, when dealing with adversity students will flip to social assist systems to buffer this stress. Firms like IFTTT and Zapier are at the heart of this. College students obviously use ICT to communicate with buddies and, as this study has proven, is the preferred method of communication.
Identification and intimacy crises and their relationship to Web dependence amongst college students. Student Sleep: Relationship to Well being and Educational Efficiency. Once they want to know one thing they Google it. After they wish to purchase something they go to Amazon.


गुरु जम्भेश्वर विज्ञान और प्रौद्योगिकी विश्वविद्यालय ने 31 पदों पर Asst Professor, Asst Librarian, Demonstrator की भर्ती के लिए एक अधिसूचना जारी की है। इच्छुक उम्मीदवार 11 फरवरी 2019 से आवेदन कर सकते हैं। आवेदन करने से पहले पूर्ण अधिसूचना पढ़ें ।। Guru Jambheshwar University of Science and Technology vacancy notification 2019 for Asst Professor, Asst Librarian, Demonstrator 31 posts.
1. Candidates should have Degree, Post Graduate.
Candidates will be selected based on Practical Exam & Interview.


The forestry division is a group of committed tree and landscape care personnel that tend to the health and beauty of the green public landscape. The work performed by this division includes, but is not limited to, trimming of healthy trees, removal of dead or diseased trees, planting of new trees and brush clearing along public roadways.
It has also been scientifically recognized that there is a strong connection with the more trees a neighborhood has, the lower its crime rates….the reason for that is disputed but up to you to consider!


Give your kitchen a colourful lift with the Umbra Joey Ceramic Soap Pump Avocado!
This modern green soap dispenser boasts a visually appealing design aesthetic, and is just the ticket for adding some colourful style to your kitchen, or even your bathroom! The name "Joey" originates from the integrated kangaroo-style pouch that holds a handy dish scrubby (which is included). Washing up isn't our favourite activity but we think adding some fun colour to proceedings will make it less of a chore!
A wonderful way of adding a subtle cheerful accent colour to your sink space, Joey is a perfect match for Umbra's other avocado washing up accessories - as seen on the right hand side of the page! But if avocado green isn't to your tastes, Joey comes in a range of other colours, including red, black and teal!


photography. It's choc-a-bloc with useful tips and meal ideas.
before and in the morning you can just grab and go!
or even coconut milk. Cover and refrigerate overnight.
or granola and dig in. Thank me (and Food52) later!How do you like your oats?


text:
6 Reasons to Wear Power Suits in 2020 Inspired by The Politician's Laura Dreyfuss
Here are our favorites from the 2020 Golden Globe-nominated series.
By Chelsea Greenwood
While the star of Netflix's new dramedy series The Politician is undoubtedly Ben Platt – known for his role as Evan Hansen on Broadway's Dear Evan Hansen – we think another member of the cast comes pretty close to stealing his spotlight.
That's McAfee Westbrook, played by Laura Dreyfuss, who also happened to co-star alongside Platt in Dear Evan Hansen. She plays a high school student who's hell-bent on getting her friend Payton Hobart (Platt) elected student body president as part of his lifelong plan to ultimately become POTUS. As a member of his dedicated campaign staff of fellow students, McAfee is a fast-talking, no-nonsense master strategist who seems to have all the answers.
Perfectly underscoring her personality and motivations is her wardrobe of bold power suits, which she wears the majority of the time and typically pairs with perfectly tousled hair, minimal makeup, and a wide variety of shoes.
Costume designer Claire Parkinson told Teen Vogue that, after meeting and discussing the character with showrunner Ryan Murphy, they decided to dress McAfee in a uniform-like style that fits with the satirical nature of the show.
"Although McAfee is often very serious and her main agenda is to get Payton elected as high school president, we still wanted her style to be elevated and bold, like her character," she said. "We chose vibrant colors and many different shapes, with some standout accessories to show that she means business but still can have fun with her style."
And while each power suit is a fashion statement in its own right, we definitely have our favorites. Here they are, ranked. Hopefully these give you the inspiration to wear full-on power suits in 2020, it surely has for us/
5. Pink double-breasted suit (Episode 6)
Courtesy of Netflix
In this scene, McAfee sits with Payton's vice president (spoiler alert: he becomes president) and her secret girlfriend, Skye Leighton (Rahne Jones), who's doubting Payton's vision as he addresses the school board.
McAfee's ensemble – the wide-leg pants, bubblegum pink hue, and loud button-up blouse – scream 1970s in the best way possible. Even if she's losing confidence in her leading man, her outfit is absolutely self-assured. The suit comprises the Zara Double Breasted Blazer and Coloured Wide Leg Trousers, according to [Shop Your TV] (https://www.shopyourtv.com/2019/10/01/the-politician-season-1-episode-6-mcafees-pink-suit/).
4. Mustard wide-leg suit (Episode 2)
McAfee sits with Payton and another of their campaign staffers, James Sullivan (Theo Germaine), during one of the group's first moments of doubt: when they find out that their choice for student body vice president, Infinity Jackson (Zoey Deutch), has a secret that could destroy their chances of election.
McAfee's mustard suit brings to mind the fashion muses that Parkinson told *W* (https://www.wmagazine.com/story/the-politician-costume-designers-fashion-interview) she referenced in creating the character's costumes, including Gloria Steinem, Diane Keaton, and Katharine Hepburn. The double-breasted jacket (a recurring theme) pairs with matching ASOS DESIGN Mustard Wide Leg Pants, according to Shop Your [TV] (https://www.shopyourtv.com/2019/09/27/the-politician-season-1-episode-2-mcafees-mustard-trousers/), featuring a high rise and a wide-cut leg.
Finishing off the outfit are a couple of gold rings, chunky dad sneakers, and a patterned long-sleeve shirt that might be one of the vintage '70s blouses that Parkinson said she enjoyed pairing with McAfee's contemporary suits.
3. Peach single-breasted suit (Episode 1)
In the pilot episode, Payton's team – including his girlfriend, Alice Charles (Julia Schlaepfer) – gathers to discuss campaign strategy, and McAfee is still firmly in his camp.
Her sherbet-colored ensemble is pretty tone-on-tone, with its peach single-breasted suit, pink top, and rose gold aviators, until you get to her shoes. The Comme des Garcons Play Black Converse Edition Half Heart Chuck 70 High Sneakers, according to Shop Your TV, lend a punky toughness and street-style edge to the outfit, saying: "Don't mess with me."
2. Print short suit (Episode 4)
This retro-inspired, print short suit reveals a marked change in McAfee. It's the first time we see her in shorts, revealing a bit of skin, and also the first time we see her expressing feelings for Skye. McAfee is doing something for herself for once – and not for Payton.
Parkinson said: "We loved using the short suit with Margiela platforms for when she's first seen going to Skye's house. She gets out of her fabulous vintage VW bug wearing a seemingly matching tan short suit with sky-high platforms – this suit was a bit more relaxed, and she's less covered up."
The short suit is the Free People Barb Suit Set, according to Shop Your TV, and the shoes are MM6 Maison Margiela Suede Platform Boots, according to Shop Your TV.
1. Neon green suit (Episode 1)
The neon green Rachel Comey suit from the pilot is Parkinson's favorite power suit of the bunch, and we agree. In this outfit, McAfee stands out like a highlighter on a page. There's no missing her.
"We chose this suit for many important scenes in Episode 1 to show she means business while looking fabulous," Parkinson said.
Complementing the knit, cropped-leg suit are carefully chosen accessories, she said: "My assistant costume designer and I found the Celine loafers on sale, and they were such a key component to the head-to-toe look. She wore a Dries Van Noten top under the suit."
The finished look is equal parts retro, modern, bold, chic – and 100% McAfee.
Keywordstrendingsuit
These Are the Valentine's Day Outfit Ideas We're Loving
To love and be loved…in a great outfit!
Addison Rae Channeled Marilyn Monroe With This Look
She even got the pose down.
By Vadana Pawa
Hailey Bieber's New Hair Is the Perfect Complement to Her Style
See her best looks since her major chop.
By Shama Nasinde
Sadie Sink Makes the Case for Wide-Leg Jeans
In true Gen Z fashion.
By Mayra Peralta


text:
Q: Transform array elements to parent and children Want to transform array from single layer to nested.
So if the element has field parentId="3" it must be added to children field of that element who has id="3"
For example "Late" must be a children of "Coffee", and "Nested Late" must be a children of "Late".
Example:
input data:
const data = [
    {
      id: "2",
      name: "Coffee",
      parentId: null,
    },
    {
      id: "3",
      name: "Late",
      parentId: "2",
    },
    {
      id: "6",
      name: "Nested Late",
      parentId: "3",
    },
    {
      id: "4",
      name: "Books",
      parentId: null,
    },
    {
      id: "5",
      name: "Nested Book",
      parentId: "4",
    }
  ];

output:
const result = [
  {
    id: "2",
    name: "Coffee",
    parentId: null,
    children: [
      {
        id: "3",
        name: "Late",
        parentId: "2",
        children: [
          {
            id: "6",
            name: "Nested Late",
            parentId: "3",
          },
        ],
      }
    ],
  },
  {
    id: "4",
    name: "Books",
    parentId: null,
    children: [
      {
        id: "5",
        name: "Nested Book",
        parentId: "4",
      },
    ],
  },
];


text:
Q: Does the size, and distance between, expanding ultracold gas clouds depend on the populations in each cloud? Say you had some $^{87}$Rb atoms at a few kelvin and you knew that the atoms were
distributed among the magnetic sub-levels of a hyperfine manifold. You want to know the relative population of each by use of a linear magnetic field gradient. Does the population of each of the states determine the size of the clouds and the distances between them during the expansion?
I'm thinking that the interactions between the atoms will make the states with more atoms larger but the distance between the expanding cloud centers should be the same?


As some of you may know, I contribute to Pop Goes the Library, a blog about pop culture and libraries. I recently had the privilege of interviewing Cathy Belben, a new staff writer at Veronica Mars. You can read the article here. Teaser for the interview: Cathy was a high school librarian who had never written a script for TV before joining VM.
For those of you who haven't yet started watching one of the best written and acted shows on TV: the new season starts September 21st and the DVDs for the first season will be available on October 11th.


text:
Laura Nalbandian: From Disillusionment to Inspiration – Neptune in the Natal Chart
Don't miss this opportunity to learn from Laura Nalbandian as she appears via webinar! Note Start Time is 1 PM (Pacific)
More Details at: https://ncgrsanfrancisco.org
Feb 18, 2023 01:00 PM in Pacific Time (US and Canada)
Hi there, You are invited to a Zoom webinar. When: Feb 18, 2023 01:00 PM Pacific Time (US and Canada) Topic: Laura Nalbandian: From Disillusionment to Inspiration – Neptune in the Natal Chart Register in advance for this webinar: https://us02web.zoom.us/webinar/register/WN_w8aGV5_HQXCYRl2BV_R5Dw After registering, you will receive a confirmation email containing information about joining the webinar.


Ever since my quick visit to the Clifton Inn last summer on my way to North Carolina, I've been trying to coordinate a trip for Michael, Orvis, and I to spend some time in Charlottesville, Virginia together. Things happened to come together and we were able to spend two nights in the area en route to North Carolina earlier this month. It didn't take us long to fall pretty hard for this charming town – UVA's gorgeous colonial campus rivals that of Carolina's (which is pretty huge for two UNC alums to admit and something I think Michael is still having a hard time coming to terms with) and there's so much to see and experience throughout downtown and in the surrounding area. We could have spent an entire week in Charlottesville and just barely scratched the surface of the shops, restaurants, and scenery. In our short visit, we not only had time to hike, eat, and explore, but we also had the chance to catch up with some friends who live in the area. We had an amazing evening, conversation, and home-cooked meal with Abby and Guion, and their beautiful puppy Pyrrha even graciously extended her yard to Orvis.
We felt right at home browsing the goods at this darling shop downtown. Everything was so perfectly arranged, and it was hard to let Michael convince me that we didn't have enough room in the car for everything I wanted to bring home.
Coincidentally sharing the name on another of our favorite shops, this must-visit store is right on the mall downtown, complete with our very favorites in paper goods and the cutest office accessories. While we were here, we found a delightful little "Welcome" sign to greet guests above the door in our mudroom.
Dinner at Mas Tapas after the long trip down from Maine was such a treat! It was our first al fresco meal of the season (oh, spring in the south!) and we stuffed ourselves with delicious small plates, including tasty papas bravas, lamb shanks, quesos, and bacon-wrapped dates.
Anne at Clifton recommended eating lunch on the mall at Petit Pois, and since they allowed Orvis to accompany us (clearly our favorite way to dine), we jumped at the opportunity! We devoured our meal of mussels, risotto, and beef bourguinon while people watching, laughing at Orvis, and dreaming about future trips to Charlottesville.
On our way out of town, we grabbed a delicious bite to eat and a couple of bottles of local vineyard wine (we'll have to save vineyard tours for our next visit to Charlottesville) from Feast! in the Main Street Market. Aside from Feast! the market was packed with adorable shops, other restaurants, and patrons. It was a surprising plus to end our trip.
Driving in and out of Clifton, we kept passing by a darling ivy-clad little market at an intersection in the country. It immediately piqued our interest, and it wasn't long before we stopped into the well-branded Salt to see what this market was all about. They had literally just opened the doors for business and we had the chance to talk to the owner quite a bit about her plans for the market's future. Salt is a perfect place to grab a sandwich or salad to go, or pick up a cheese plate for a picnic on one of the many hiking trails. We look forward to visiting Salt again on our next trip to see how it's evolved within the Charlottesville community.
I'm sure that there are plenty of places to stay in the Charlottesville area, but given the comfort, beauty, and charm of Clifton, I can't imagine Michael or I going anywhere else. They're dog-friendly to boot, and Orvis loves romping around the grounds and swimming in the lake (although he's still a little hesitant around the property's resident cats).
Thomas Jefferson's home is obviously a plus for any American history buffs out there, but if you can make it to Monticello in the springtime, the cherry blossoms, tulip-lined paths, and expansive garden will easily win you over.
This trail system was recommended to us by Anne at Clifton. Running along the Thomas Jefferson Parkway, there are a number of different trails from which to choose (even a bridged path along the parkway to Monticello). We took Orvis for a hike on the Mountain Trail and quickly realized just how much the "menu" part of Map & Menu has been taking its toll on us.


The parish office is located at 5135 North Cocoa Blvd. in Cocoa, Florida. The office is open from 9:00 am until 4:00 pm Monday through Friday. If you have an emergency, call the main number (321- 632-6333). If you receive a recorded message, there is an option to redirect your call in an emergency.
For information on marriage preparation and to make an appointment to schedule a wedding, email the parish secretary (secretary@blessedsacramentcocoa.org ). You may also click on the marriage link under sacraments on this website.
To get more information about baptisms or to schedule a date for a baptism, email the parish secretary (secretary@blessedsacramentcocoa.org). You may also click on the baptism link under sacraments on this website.
To register as a parishioner fill out a parish registration form and return it to the Parish office. Parish Registration forms can be found in the back of the church at the parish office or click on the parish registration link on this website.
To find out more about or join any of our ministries, please email secretary@blessedsacramentcocoa.org and your inquiry will be forwarded appropriately.
Mass schedules for Sundays and special feasts/holy days are always posted. See the Mass times link. Upcoming events are listed on the events page. Contact email addresses and names for special events are generally noted along with the description of the event.


text:
\section{Discussion}
We proposed a new approach to address the memory issues of training iterative inverse models using invertible neural networks. This enabled us to train very deep models on a large scale imaging task which would have been impossible to do with earlier approaches. The resulting models learn to do state-of-the-art image reconstruction.
We further introduced a new invertible layer that allows us to train our deep models in a stable way. Due to it's structure our proposed layer lends itself to structured prediction tasks, and we expect it to be useful in other such tasks as well.
Because invertible neural networks have been predominantly used for unsupervised training, our approach naturally allows us to exploit these directions as well. In the future, we aim to train our models in an unsupervised or semi-supervised fashion.

\section{Introduction}

We consider the task of solving inverse problems. An inverse problem is described through a   so called forward problem that models either a real world measurement process or an auxiliary prediction task. The forward problem can be written as
\begin{align}\label{eq:inverse_problem}
    \mathbf{d} &= \mathcal{A}(\mathbf{p},\mathbf{n})
\end{align}
where $\mathbf{d}$ is the measurable data, $\mathcal{A}$ is a (non-)linear forward operator that models the measurement process, $\mathbf{p}$ is an unobserved signal of interest, and $\mathbf{n}$ is observational noise. Solving the inverse problem is then a matter of finding an inverse model $\mathbf{p} = \mathcal{A}^{-1}(\mathbf{d})$. However, if the problem is ill-posed or the forward problem is non-linear, finding $\mathcal{A}^{-1}$ is a non-trivial task. Oftentimes, it is necessary to impose assumptions about signal $\mathbf{p}$ and to solve the task in an iterative fashion \citep{Candes2006}.

\subsection{Learn to Invert}
\begin{figure}
    \centering
    \includegraphics[width=\textwidth]{figures/rim_irim_diagram.png}
    \caption{Iterative Inverse Models: Unrolled model and individual steps of RIM and i-RIM.}
    \label{fig:architecture_schematic}
\end{figure}
Many recent approached to solving inverse problems focus on models that \textit{learn to invert} the forward problem by mimicking the behaviour of an iterative optimization algorithm. Here, we will refer to models of this type as ``iterative inverse models''. Most iterative inverse models can be described through a recursive update equation of the form
\begin{align}
\mathbf{p}_{t+1}, \mathbf{s}_{t+1} = h_\phi(\mathcal{A},\mathbf{d},\mathbf{p}_t,\mathbf{s}_t)
\end{align}
where $h_\phi$ is a parametric function, $\mathbf{p}_{t}$ is an estimate of signal $\mathbf{p}$ and $\mathbf{s}_t$ is an auxiliary (memory) variable at iteration $t$, respectively. Because $h_\phi$ is an iterative model it is often interpreted as a recurrent neural network (RNN). The functional form of $h_\phi$ ultimately characterizes different approaches to iterative inverse models. Figure \ref{fig:architecture_schematic} (A) illustrates the iterative steps of such models.

Training of iterative inverse models is typically done via supervised learning. To generate training data, measurements $\mathbf{d}$ are simulated from ground truth observations $\mathbf{p}$ through a known forward model $\mathcal{A}$. The training loss then takes the form
\begin{align}\label{eq:metalearn_loss}
    \mathcal{L}(\mathbf{p};\mathbf{d},\mathcal{A},\phi) = \sum_t^{T} \omega_t \mathcal{L}\left(\mathbf{p},\hat{\mathbf{p}}_{t}(\mathcal{A},\mathbf{d},\phi)\right)
\end{align}
where $\mathbf{p}$ is the ground truth signal, $\hat{\mathbf{p}}_{t}(\mathcal{A},\mathbf{d},\phi)$ is the model estimate at iteration $t$ given data $(\mathcal{A},\mathbf{d})$ and model parameters $\phi$, and $\omega_t \in \mathbb{R}^+$ is an importance weight of the $t$-th iteration.

Models of this form have been successfully applied in several image restoration tasks on natural images
\cite{putzky2017recurrent,Chen2015,Wang2016,Zheng2015,Schmidt2014,Schmidt2016}, on sparse coding \cite{Gregor2010}, and more recently have found adoption in medical image reconstruction  \cite{lonning2019recurrent,Adler2017,Hammernik2017,Schlemper2017}, a field where sparse coding and compressed sensing have been a dominant force over the past years \cite{Candes2006,Candes2006a}.

\subsection{Invert to Learn}

In order to perform back-propagation efficiently, we are typically required to store intermediate forward activations in memory. This imposes a trade-off between model complexity and hardware memory constraints which essentially limits network depth. Since iterative inverse models are trained with back-propagation through time, they represent some of the deepest, most memory consuming models currently used. As a result, one often has to resort to very shallow models at each step of the iterative process. Here, we overcome these model limitations by presenting a memory efficient way to train very deep iterative inverse models. To do that, our approach follows the training principles presented in \citet{Gomez2017}. To save memory, the authors suggested to use reversible neural network architectures which make it unnecessary to store intermediate activations as they can be restored from post-activations. Memory complexity in this approach is $\mathcal{O}(1)$ and computational complexity is $\mathcal{O}(L)$, where $L$ is the depth of the network. In the original approach, the authors utilized pooling layers which still required storing of intermediate activations at these layers. In practice, memory cost was hence not $\mathcal{O}(1)$. Briefly after, \citet{Jacobsen2018} demonstrated that a fully invertible network inspired by RevNets \cite{Gomez2017} can perform as well as a non-invertible model on discriminative tasks, although the model was not trained with memory savings in mind. Here we will adapt invertible neural networks to allow for memory savings in the same way as in \citet{Gomez2017}. We refer to this approach as ``invertible learning''.

\subsection{Invertible Neural Networks}
Invertible Neural Networks have become popular predominantly in likelihood-based generative models that make use of the change of variable formula:
\begin{align}\label{eq:gen_like}
    p_\mathbf{x}(\mathbf{x}) = p_{\mathbf{y}}\left(f\left(\mathbf{x}\right)\right) \left| \det \left(\frac{\partial f(\mathbf{x})}{\partial \mathbf{x}^\top}\right) \right|
\end{align}
which holds for invertible $f\left(\mathbf{x}\right)$ \cite{Dinh2014}. Under a fixed distribution  $p_{\mathbf{y}}\left(\cdot\right)$, an invertible model $f\left(\mathbf{x}\right)$ can be optimized to maximize the likelihood of data $\mathbf{x}$.
\citet{Dinh2014} suggested an architecture for invertible networks in which inputs and outputs of a layer are split into two parts such that $\mathbf{x} = (\mathbf{x}_1, \mathbf{x}_2)$ and $\mathbf{y} = (\mathbf{y}_1, \mathbf{y}_2)$. Their suggested invertible layer has the form
\begin{equation}\label{eq:nice_update}
  \begin{aligned}
    \mathbf{y}_1 &= \mathbf{x}_1 \\
    \mathbf{y}_2 &= \mathbf{x}_2 + \mathcal{G}(\mathbf{y}_1)
  \end{aligned}
  \qquad \qquad
  \begin{aligned}
    \mathbf{x}_2 &= \mathbf{y}_2 - \mathcal{G}(\mathbf{y}_1) \\
    \mathbf{x}_1 &= \mathbf{y}_1
  \end{aligned}
\end{equation}
where the left-hand equations reflect the forward computation and the right-hand equation are the backward computations. This layer is similar to the layer used later in \citet{Gomez2017}.
Later work on likelihood-based generative models built on the idea of invertible networks by focusing on non-volume preserving layers \cite{Dinh2017,Kingma2018glow}. A recent work avoids the splitting from \citet{Dinh2014} and focuses instead on $f(\mathbf{x})$ that can be numerically inverted \cite{Behrmann2018invertible}.

Here, we will assume that any invertible neural network can be used as a basis for invertible learning. Apart from the fact that our approach will allow us to have significant memory savings during training, it can further allow us to leverage \eqref{eq:gen_like} for unsupervised training in the future.

\subsection{Recurrent Inference Machines}
We will use Recurrent Inference Machines (RIM) \cite{putzky2017recurrent} as a basis for our iterative inverse model. The update equations of the RIM take the form
\begin{align}
\label{eq:rim_update_equation_s}
\mathbf{s}_{t+1} &= f\left(\nabla\mathcal{D}\left(\mathbf{d}, \mathcal{A}\left(\Psi\left(\boldsymbol{\eta}_t\right)\right)\right),\boldsymbol{\eta}_{t},\mathbf{s}_{t}\right) \\
\label{eq:rim_update_equation_z}
\boldsymbol{\eta}_{t+1} &= \boldsymbol{\eta}_t + g\left(\nabla\mathcal{D}\left(\mathbf{d}, \mathcal{A}\left(\Psi\left(\boldsymbol{\eta}_t\right)\right)\right),\boldsymbol{\eta}_{t},\mathbf{s}_{t+1}\right),
\end{align}
with $\mathbf{p}_t = \Psi(\boldsymbol{\eta}_t)$, where $\Psi$ is a link function, and $\mathcal{D}\left(\mathbf{d}, \mathcal{A}\left(\Psi\left(\boldsymbol{\eta}_t\right)\right)\right)$ is a data consistency term which ensures that estimates of $\mathbf{p}_t$ stay true to the measured data $\mathbf{d}$ under the forward model $\mathcal{A}$. Below, we will call $(\boldsymbol{\eta}_t,\mathbf{s}_t)$ the machine state at time t. The benefit of the RIM is that it simultaneously learns iterative inference and it implicitly learns a prior over $\mathbf{p}$. The update equations are general enough that they allow us to use any network for $h_{\phi}$. Unfortunately, even if $h_{\phi}$ was invertible, the RIM in it's current form is not. We will show later how a simple modification of these equations can make the whole iterative process invertible. An illustration of the update block for the machine state can be found in figure \ref{fig:architecture_schematic} (B).

\subsection{Contribution}
In this work, we marry iterative inverse models with the concept of invertible learning. This leads to the following contributions:
\begin{enumerate}[wide, labelwidth=!, labelindent=0pt]
    \item \textbf{The first iterative inverse model that is fully invertible.} This allows us to overcome memory constraints during training with invertible learning \cite{Gomez2017}. It will further allow for semi- and un-supervised training in the future \cite{Dinh2014}.
    \item \textbf{Stable invertible learning of very deep models.} In practice, invertible learning can be unstable due to numerical errors that accumulate in very deep networks \cite{Gomez2017}. In our experience, common invertible layers \cite{Dinh2014,Dinh2017,Kingma2018glow} suffered from this problem. We give intuitions why these layers might introduce training instabilities, and present a new layer that addresses these issues and enables stable invertible learning of very deep networks (400 layers).
    \item \textbf{Scale to large observations.} We demonstrate in experiments that our model can be trained on large volumes in MRI (3d). Previous iterative inverse models were only able to perform reconstruction on 2d slices. For data that has been acquired with 3d sequences, however, these approaches are not feasible anymore. Our approach overcomes this issue. This result has implications in other domains with large observations such as synthesis imaging in radio astronomy.
\end{enumerate}

\subsubsection*{Acknowledgements}
The authors are grateful for helpful comments from Matthan Caan, Clemens Kornd\"{o}rfer, J\"{o}rn Jacobsen, Mijung Park, Nicola Pezzotti, and Isabel Valera.

Patrick Putzky is supported by the Netherlands Organisation for Scientific Research (NWO) and the Netherlands Institute for Radio Astronomy (ASTRON) through the big bang, big data grant.
\bibliographystyle{unsrtnat}

\section{Method}
Our approach consists of two components which we will describe in the following section. (1) We present a simple way to modify Recurrent Inference Machines (RIM) \cite{putzky2017recurrent} such that they become fully invertible. We call this model ``invertible Recurrent Inference Machines'' (i-RIM). (2) We present a new invertible layer with modified ideas from \citet{Kingma2018glow} and \citet{Dinh2014} that allows stable invertible learning of very deep i-RIMs. We briefly discuss why invertible learning with conventional layers \citep{Dinh2014, Dinh2017, Kingma2018glow} can be unstable. An implementation of our approach can be found at \url{https://github.com/pputzky/invertible_rim}.

\subsection{Invertible Recurrent Inference Machines}
In the following, we will assume that we can construct an invertible neural network $h(\cdot)$ with memory complexity $\mathcal{O}(1)$ during training using the approach in \citet{Gomez2017}. That is, if the network can be inverted layer-wise such that
\begin{align}
    &h = h^L \circ h^{L-1} \circ \dots \circ h^1 \\
    &h^{-1} = (h^1)^{-1} \circ (h^2)^{-1} \circ \dots \circ (h^L)^{-1}
\end{align}
where $h^l$ is the $l$-th layer, and $(h^l)^{-1}$ is it's inverse, we can use invertible learning do back-propagation without storing activations in a computationally efficient way.

Even though we might have an invertible $h(\cdot)$, the RIM in the formulation given in \eqref{eq:rim_update_equation_z} and \eqref{eq:rim_update_equation_s} cannot trivially be made invertible. One issue is that if naively implemented, $h(\cdot)$ takes three inputs but has only two outputs as $\nabla\mathcal{D}(\mathbf{d}, \mathcal{A}\Psi\boldsymbol{\eta}_t)$ is not part of the machine state $(\boldsymbol{\eta}_t,\mathbf{s}_t)$. With this, $h(\cdot)$ would have to increase in size with the number of iterations of the model in order to stay fully invertible. The next issue is that the incremental update of $\boldsymbol{\eta}_t$ in \eqref{eq:rim_update_equation_z} is conditioned on $\boldsymbol{\eta}_t$ itself, which prevents us to use the trick from \eqref{eq:nice_update}. One solution would be to save all intermediate $\eta_t$ but then we would still have memory complexity $\mathcal{O}(T) < \mathcal{O}(T*L)$ which is an improvement but still too restrictive for large scale data (compare \ref{sec:results}, i-RIM 3D). Alternatively, \citet{Behrmann2018invertible} show that \eqref{eq:rim_update_equation_z} could be numerically inverted if $\text{Lip}(h) < 1$ in $\boldsymbol{\eta}$. In order to satisfy this condition, this would involve not only restricting $h(\cdot)$ but also on $\mathcal{A}$, $\Psi$, and $\mathcal{D}(\cdot,\cdot)$. Since we want our method to be usable with any type of inverse problem described in \eqref{eq:inverse_problem} such an approach becomes infeasible. Further, for very deep $h(\cdot)$ numerical inversion would put a high computational burden on the training procedure.

We are therefore looking for a way to make the update equations of the RIM trivially invertible. To do that we use the same trick as in \eqref{eq:nice_update}. The update equations of our invertible Recurrent Inference Machines (i-RIM) take the form (left - forward step; right - reverse step):
\begin{equation}
  \begin{aligned}
    &\z^\prime_{t} = \boldsymbol{\eta}_t\\
&\s^\prime_{t} = \mathbf{s}_{t} + g_t(\nabla\mathcal{D}\left(\mathbf{d}, \mathcal{A}\left(\Psi\left(\z^\prime_t\right)\right)\right)) \\ &\boldsymbol{\eta}_{t+1}, \mathbf{s}_{t+1} = h_t(\z^\prime_t, \s^\prime_{t})
  \end{aligned}
  \qquad \qquad
\begin{aligned}
    &\z^\prime_{t}, \s^\prime_{t} = h_t^{-1}(\boldsymbol{\eta}_{t+1}, \mathbf{s}_{t+1}) \\
    &\mathbf{s}_{t} = \s^\prime_{t} - g_t(\nabla\mathcal{D}\left(\mathbf{d}, \mathcal{A}\left(\Psi\left(\z^\prime_t\right)\right)\right)) \\
    &\boldsymbol{\eta}_t = \z^\prime_{t}
\end{aligned}
\end{equation}
where we do not require weight sharing over iterations $t$ in $g_t$ or $h_t$, respectively. As can be seen, the update from $(\boldsymbol{\eta}_t,\mathbf{s}_t)$ to $(\z^\prime_t,\s^\prime_t)$ is reminiscent of \eqref{eq:nice_update}, we assume that $h(\cdot)$ is an invertible function. Given $h(\cdot)$ can be inverted layer-wise as above, we can train an i-RIM model using invertible learning with memory complexity $\mathcal{O}(1)$. We have visualised the forward and reverse updates of the machine state in an i-RIM in figure \ref{fig:architecture_schematic}(C) \& (D).

\subsection{An Invertible Layer with Orthogonal 1x1 convolutions}
Here, we introduce a new invertible layer that we will use to form $h(\cdot)$. Much of the recent work on invertible neural networks has focused on modifying the invertible layer of \citet{Dinh2014} in order to improve generative modeling. \citet{Dinh2017} proposed a non-volume preserving layer that can still be easily inverted:
\begin{equation}\label{eq:real_nvp}
  \begin{aligned}
    \mathbf{y}_1 &= \mathbf{x}_1 \\
    \mathbf{y}_2 &= \mathbf{x}_2 \odot \exp(\mathcal{F}(\mathbf{y}_1)) + \mathcal{G}(\mathbf{y}_1)
  \end{aligned}
  \qquad \qquad
  \begin{aligned}
    \mathbf{x}_2 &= \left(\mathbf{y}_2 - \mathcal{G}(\mathbf{y}_1) \right) \odot \exp(-\mathcal{F}(\mathbf{y}_1)) \\
    \mathbf{x}_1 &= \mathbf{y}_1
  \end{aligned}
\end{equation}
While improving over the volume preserving layer in \citet{Dinh2014}, the method still required manual splitting of layer activations using a hand-chosen mask. \citet{Kingma2018glow} addressed this issue by introducing invertible $1\times1$ convolutions to embed the activations before using the affine layer from \citet{Dinh2017}. The idea behind this approach is that this convolution can learn to permute the signal across the channel dimension to make the splitting a parametric approach. Following this, \citet{hoogeboom2019emerging} introduced a more general form of invertible convolutions which operate not only on the channel dimension but also on spatial dimensions.

We have tried to use the invertible layer of \citet{Kingma2018glow} in our i-RIM but without success. We suspect three possible causes of this issue which we will use as motivation for our proposed invertible layer:
\begin{enumerate}[wide, labelwidth=!, labelindent=0pt]
    \item \textbf{Channel permutations} Channel ordering has a semantic meaning in the i-RIM (see \eqref{eq:irim_h}) which it does not have a priori in the above described likelihood-based generative models. Further, a permutation in layer $l$ will affect all channel orderings in down-stream layers $k>l$. Both factors may harm the error landscape.
    \item \textbf{Exponential gate} The rescaling term $\exp(\mathcal{F}(\mathbf{x}))$ in \eqref{eq:real_nvp} can make inversions numerically unstable if not properly taken care of.
    \item \textbf{Invertible 1x1 convolutions} Without any restrictions on the eigenvalues of the convolution it is possible that it will cause vanishing or exploding gradients during training with back-propagation.
\end{enumerate}

For the of the i-RIM, we have found an invertible layer that addresses all the potential issues mentioned above, is simple to implement, and leads to good performance as can be seen later. Our invertible layer has the following computational steps:
\begin{align}
    \x^\prime &= \mathbf{U} \mathbf{x} \\
    \y^\prime_1 &= \x^\prime_1 \\
    \y^\prime_2 &= \x^\prime_2  + \mathcal{G}(\x^\prime_1)\\
    \mathbf{y} &= \mathbf{U}^\top\y^\prime \label{eq:orth_backproject}
\end{align}
where $\x^\prime = (\x^\prime_1,\x^\prime_2)$ and $\y^\prime = (\y^\prime_1,\y^\prime_2)$, and $\mathbf{U}$ is an orthogonal $1\times1$ convolution which is key to our invertible layer. Recall, the motivation for an invertible $1\times1$ convolution in \citet{Kingma2018glow} was to learn a parametric permutation over channels in order to have a suitable embedding for the following affine transformation from \citet{Dinh2014,Dinh2017}. An orthogonal $1\times1$ convolution is sufficient to implement this kind of permutation but will not cause any vanishing or exploding gradients during back-propagation since all eigenvalues of the matrix are $1$. Further, $\mathbf{U}$ can be trivially inverted with $\mathbf{U}^{-1} = \mathbf{U}^\top$ which will reduce computational cost in our training procedure as we require layer inversions at every optimization step. Below, we will show how to construct an orthogonal $1\times1$ convolution. Another feature of our invertible layer is \eqref{eq:orth_backproject} in which we project the outputs of the affine layer back to the original basis of $\mathbf{x}$ using $\mathbf{U}^\top$. This means that our $1\times1$ convolution will act only locally on it's layer while undoing the permutation for downstream layers. A schematic of our layer and it's inverse can be found in figure \ref{fig:flow}.Here, we will omit the inversion of our layer for brevity and refer the reader to the supplement.
\begin{figure}[t]
    \centering
    \includegraphics[width=0.8\textwidth]{figures/flow_schematic.png}
    \caption{Illustration of the layer used in this work. (A): Invertible layer with orthogonal $1\times1$ convolutional embedding. (B): Function $\mathcal{G}$ used in each invertible layer.}
    \label{fig:flow}
\end{figure}

\subsubsection{Orthogonal 1x1 convolution}
A $1\times1$ convolution can be implemented through using a $k\times k$ matrix that is reshaped into a convolutional filter and then used in a convolution operation \cite{Kingma2018glow}. In order to guarantee that this matrix is orthogonal we use the method utilized in \citet{tomczak2016improving} and \citet{hoogeboom2019emerging}.
Any orthogonal $k\times k$ matrix $\mathbf{U}$ can be constructed from a series of Householder reflections such that
\begin{align}
    \mathbf{U} = \mathbf{H}_K \mathbf{H}_{K-1} \dots \mathbf{H}_1
\end{align}
where $\mathbf{H}_k$ is a Householder reflection with
$   \mathbf{H}_k = \mathbf{I} - 2 \frac{\mathbf{v}_k \mathbf{v}_k^\top}{\lVert\mathbf{v}_k\rVert^2}$,
and $\mathbf{v}_k$ is a vector which is orthogonal to the reflection hyperplane. In our approach, the vectors $\mathbf{v}_k$ represent the parameters to be optimized. Because the Householder matrix is itself orthogonal, a matrix $\mathbf{U}_D$ that is constructed from a subset $D < K$ of Householder reflections such that
    $\mathbf{U}_D = \mathbf{H}_D \mathbf{H}_{D-1} \dots \mathbf{H}_1$
is still orthogonal. We will use this fact to reduce the amount of parameters necessary to train on the one hand and to reduce the cost of constructing $\mathbf{U}_D$ on the other.  For our experiments we chose $D=3$.

\subsubsection{Residual Block with Spatial Downsampling}
In this work we utilize a residual block that is inspired by the multiscale approach used in \citet{Jacobsen2018}. However, instead of shuffling pixels in the machine state, we perform this action in the residual block. Each block consists of three convolutional layers. The first layer performs a spatial downsampling operation of factor $d$, i.e. the convolutional filter will have size $d$ in every dimension, and we perform a strided convolution with stride $d$. This is equivalent to shuffeling pixels in a $d\times d$ patch into different channels of the same pixel followed by a $1\times1$ convolution. The second convolutional layer is a simple $3\times3$ convolution with stride $1$. And the last convolutional layer reverses the spatial downsampling operation with a transpose convolution. At the output we have found that a Gated Linear Unit \cite{Dauphin2017} guarantees stable training without the need for any special weight initializations. We use weight normalisation \cite{Salimans2016} for all convolutional weights in the block and we disable the bias term for the last convolution. Our residual block has two parameters: $d$ for the spatial downsampling factor, and $k$ for the number of channels in the hidden layers. An illustration of our residual block can be found in figure \ref{fig:flow}.

\subsection{Related Work}
Recently, \citet{Ardizzone2018analyzing} proposed another approach of modeling inverse problems with invertible networks. In their work, however, the authors assume that the forward problem is unknown, and possibly as difficult as the inverse problem. In our case the forward problem is typically much easier to solve than the inverse problem. The authors suggest to train the network bi-directionally which could potentially help our approach as well.
The RIM has found successful applications to several imaging problems in MRI \cite{lonning2018recurrent,lonning2019recurrent} and radio astronomy
\cite{morningstar2018analyzing,morningstar2019data}. We expect that our presented results can be translated to other applications of the RIM as well.

\section{Experiments}
We evaluate our approach on a public data set for accelerated MRI reconstruction that is part of the so called fastMRI challenge \cite{zbontar2018fastmri}. Comparisons are made between the U-Net baseline from \citet{zbontar2018fastmri}, an RIM \cite{putzky2017recurrent,lonning2019recurrent}, and an i-RIM, all operating on single 2D slices. To explore future directions and push the memory benefits of our approach to the limit we also trained a 3D i-RIM. An improvement on the results presented below can be found in \citet{putzky2019irim}.

\subsection{Accelerated MRI}
The problem in accelerated Magnetic Resonance Imaging (MRI) can be described as a linear measurement problem of the form
\begin{align}\label{eq:mri_measurement}
    \mathbf{d} = \mathbf{P}\mathscr{F}\boldsymbol{\eta} + \mathbf{n}
\end{align}
where $\mathbf{P} \in \mathbb{R}^{m \times n}$ is a sub-sampling matrix, $\mathscr{F}$ is a Fourier transform, $\mathbf{d} \in \mathbb{C}^m$ is the sub-sampled K-space data, and $\boldsymbol{\eta} \in \mathbb{C}^n$ is an image. Here, we assume that the data has been measured on a single coil, hence the measurement equation leaves out coil sensitivity models. This corresponds to the 'Single-coil task' in the fastMRI challenge \cite{zbontar2018fastmri}. Further, we set $\Psi$ to be the identity function.

\subsection{Data}
All of our experiments were run on the single-coil data from \citet{zbontar2018fastmri}. The data set consists of $973$ volumes or $34,742$ slices in the training set, $199$ volumes or $7,135$ slices in the validation set, and $108$ volumes or $3,903$ slices in the test set. While training and validation sets are both fully sampled, the test set is only sub-sampled and performance has to be evaluated through the fastMRI website \footnote{\url{http://fastmri.org}}. All volumes in the data set have vastly different sizes. For mini-batch training we therefore reduced the size of image slices to $480\times320$ (2D models) and volume size to $32\times480\times320$ (3d model). Not all training volumes had $32$ slices and hence were excluded for training the 3D model. During validation and test, we did not reduce the size of slices and volumes for reconstruction. During training, we simulated sub-sampled K-space data using \eqref{eq:mri_measurement}. As sub-sampling masks we used the masks from the test set ($108$ distinct masks) in order to simulate the corruption process in the test set. For validation, we generated sub-sampling masks in the way described in \citet{zbontar2018fastmri}. Performance was evaluated on the central $320\times320$ portion of each image slice on magnitude images as in \citet{zbontar2018fastmri}. For evaluation, all slices were treated independently.
\subsection{Training}
\begin{table} \label{tab:memory}
\caption{Comparison of memory consumption during training and testing.}
\label{tab:memory}
\resizebox{\textwidth}{!}{
\begin{tabular}{@{}lccc@{}}
\toprule
 &  RIM & i-RIM 2D & i-RIM 3D \\
\cmidrule(r){2-2} \cmidrule(r){3-4}
Size Machine State $(\boldsymbol{\eta},\mathbf{s})$ (CDHW) & $130\times1\times480\times320$ &
$64\times1\times480\times320$ &
$64\times32\times480\times320$\\
Memory Machine State $(\boldsymbol{\eta},\mathbf{s})$ (in GB) & $0.079$ &$0.039$ & $1.258$\\
Number of steps $T$& $1/4/8$ & $1/4/8$ & $1/4/8$ \\
Network Depth (\#layers) & $5/20/40$ & $50/200/400$ & $50/200/400$ \\
Memory during Testing (in GB) & $0.60$ / $0.65$ / $0.65$ & $0.20$ / $0.24$ / $0.31$ & $5.87$/$6.03$ / $6.25$\\
Memory during Training (in GB) & $2.65$ / $6.01$ / $10.49$ & $2.47$ / $2.49$ / $2.51$ & $11.51$ / $11.76$ / $11.89$ \\
\bottomrule
\end{tabular}
}
\end{table}
\begin{table}[t]
\centering
\caption{Reconstruction performance on validation and test data from the fastMRI challenge \cite{zbontar2018fastmri} under different metrics. NMSE - normalized mean-squared-error (lower is better); PSNR - peak signal-to-noise ratio (higher is better); SSIM - structural similarity index \cite{Wang2004} (higher is better).}
\begin{tabular}{@{}lcccccc@{}}
\toprule
& \multicolumn{3}{c}{4x Acceleration}  & \multicolumn{3}{c}{8x Acceleration}           \\
\cmidrule(r){2-4} \cmidrule(r){5-7}
\textbf{Validation} & NMSE & PSNR & SSIM & NMSE & PSNR & SSIM \\
\cmidrule(r){1-1} \cmidrule(r){2-4} \cmidrule(r){5-7}
U-Net \cite{zbontar2018fastmri} & $0.0342$ & $31.91$ & $0.722 $ & $0.0482$ & $29.98$ & $0.656$ \\
RIM & $0.0332$ & $32.24$ & $0.725$ & $0.0484$ & $30.03$ & $0.656$ \\
i-RIM 2D & $\mathbf{0.0316}$& $\mathbf{32.55}$& $\mathbf{0.734}$& $\mathbf{0.0429}$ & $\mathbf{30.76}$ & $\mathbf{0.669}$\\
i-RIM 3D & $0.0322$ &$32.39$& $0.731$ & $0.0435$& $30.66$ & $0.667$\\ \midrule
\textbf{Test} & NMSE & PSNR & SSIM & NMSE & PSNR & SSIM \\
\cmidrule(r){1-1} \cmidrule(r){2-4} \cmidrule(r){5-7}
U-Net \cite{zbontar2018fastmri} & $0.0320$ & $32.22$ & $0.754$ & $0.0480$ & $29.45$ & $0.651$ \\
RIM & $0.0270$& $33.39$ & $0.759$ & $0.0458$& $29.71$ & $0.650$\\
i-RIM 2D & $\mathbf{0.0255}$ & $\mathbf{33.72}$ & $\mathbf{0.767}$ & $\mathbf{0.0408}$ & $\mathbf{30.41}$ & $\mathbf{0.664}$\\
i-RIM 3D & $0.0261$ & $33.54$ & $0.764$ & $0.0413$ & $30.34$ & $0.662$\\
\bottomrule
\end{tabular
\label{tab:fastmri}
\end{table}
For the Unet, we followed the training protocol from \citet{zbontar2018fastmri}. All other models were trained to reconstruct a complex valued signal. Real and imaginary parts were treated as separate channels as done in \citet{lonning2018recurrent}. The machine state $(\boldsymbol{\eta}_0,\mathbf{s}_0)$ was initialized with zeros for all RIM and i-RIM models. As training loss, we chose the normalized mean squared error (NMSE) which showed overall best performance. To regularize training we masked model estimate $\hat{\mathbf{x}}$ and target $\mathbf{x}$ with the same random sub-sampling mask $\mathbf{m}$ such that
\begin{align}
    \mathcal{L}^*(\hat{\mathbf{x}}) = NMSE(\mathbf{m}\odot\hat{\mathbf{x}}, \mathbf{m}\odot\mathbf{x})
\end{align}
This is a simple way to add gradient noise in cases where the number of pixels is very large. We chose a sub-sampling factor of $0.01$, i.e. on average $1\%$ of pixels (voxels) from each sample were used during a back-propagation step. All iterative models were trained on $8$ inference steps.

For the RIM we used a similar model as in \citet{lonning2018recurrent}. The model consists of three convolutional layers and two gated recurrent units (GRU) \cite{Chung2014} with $64$ hidden channels each. During training the loss was averaged across all times steps. For the i-RIM, we chose similar architectures for the 2D and 3D models, respectively. The models consisted of $10$ invertible layers with a fanned downsampling structure at each time step, no parameter sharing was applied across time steps, the loss was only evaluated on the last time step. The only difference between 2D and 3D model was that the former used 2D convolutions and the latter used 3d convolutions. More details on the model architectures can be found in Appendix B. As a model for $g_t$ we chose for simplicity
\begin{align}
g_t(\nabla\mathcal{D}\left(\mathbf{d}, \mathcal{A}\left(\Psi\left(\z^\prime_t\right)\right)\right)) = \begin{pmatrix}\nabla\mathcal{D}\left(\mathbf{d}, \mathcal{A}\left(\Psi\left(\z^\prime_t\right)\right)\right) \\ \mathbf{0}_{D-2}\end{pmatrix}
\end{align}
where $\mathbf{0}_{D-2}$ is a zero-filled tensor with $D-2$ number of channels with $D$ the number of channels in $\mathbf{s}_t$, which is the simplest model for $g$ we could use. The gradient information will be mixed in the downstream processing of $h$. We chose the number of channels in the machines state $(\boldsymbol{\eta},\mathbf{s})$ to be $64$
\subsection{Results} \label{sec:results}
The motivation of our work was to reduce memory consumption of iterative inverse models. In order to emphasize the memory savings achieved with our approach we compare memory consumption of each model in table \ref{tab:memory}. Shown are data for the machine state, and memory consumption during training and testing on a single GPU. As can be seen, for the 3D model we have a machine state that occupies more that $1.25$GB of memory ($7.5\%$ of available memory in a $16$GB GPU). It would have been impossible to train an RIM with this machine state given current hardware. Also note that memory consumption is mostly independent of network depth for both i-RIM models. A small increase in memory consumption is due to the increase of the number of parameters in a deeper model. We have thus introduced a model for which network depth becomes mostly a computational consideration.

We compared all models on the validation and test set using the metrics suggested in \citet{zbontar2018fastmri}. A summary of this comparison can be found in table \ref{tab:fastmri}. Both i-RIM models consistently outperform the baselines. At the time of this writing, all three models sit on top of the challenge's Single-Coil leaderboard.\footnote{\url{http://fastmri.org/leaderboards}, Team Name: NeurIPS\_Anon; Model aliases: RIM - model\_a, i-RIM 2D - model\_b, i-RIM 3D - model\_c. See Supplement for screenshot.} The i-RIM 3D shows almost as good performance as it's 2D counterpart and we believe that with more engineering and longer training it has the potential to outperform the 2D model. A qualitative assessment of reconstructions of a single slice can be found in figure \ref{fig:fastmri}. We chose this slice because it contains a lot of details which emphasize the differences across models.

\begin{figure}[t]
    \label{fig:fastmri}
    \centering
    \subfloat[Target Image]{
        \includegraphics[width=1in]{result_images/target.png}
    }
    \subfloat[U-Net 4x]{
        \includegraphics[width=1in]{result_images/unet_acc_4.png}
    }
    \subfloat[RIM 4x]{
        \includegraphics[width=1in]{result_images/rim_acc_4.png}
    }
    \subfloat[i-RIM 4x]{
        \includegraphics[width=1in]{result_images/irim_acc_4.png}
    }
    \subfloat[i-RIM 3D 4x]{
        \includegraphics[width=1in]{result_images/irim3D_acc_4.png}
    } \\
    \subfloat[Target Image]{
        \includegraphics[width=1in]{result_images/target.png}
    }
    \subfloat[U-Net 8x]{
        \includegraphics[width=1in]{result_images/unet_acc_8.png}
    }
    \subfloat[RIM 8x]{
        \includegraphics[width=1in]{result_images/rim_acc_8.png}
    }
    \subfloat[i-RIM 8x]{
        \includegraphics[width=1in]{result_images/irim_acc_8.png}
    }
    \subfloat[i-RIM 3D 8x]{
        \includegraphics[width=1in]{result_images/irim3D_acc_8.png}
    }
\caption{Reconstructions of a central slice in volume 'file1001458.h5' from the validation set. Top: 4x acceleration. Bottom: 8x acceleration. Zoom in for better viewing experience.}
    \label{fig:fastmri}
\end{figure}

redpajama_set_name: RedPajamaArXiv


Wario Ware is a Party, Parody Song Lyrics of Xuxa, "A Vida é uma Festa/La vida es una fiesta/Life is a party"
Song Parodies -> "Wario Ware is a Party"
"A Vida é uma Festa/La vida es una fiesta/Life is a party"
"Wario Ware is a Party"
Ah! Wario Ware is a party!
but All for one is oh-so-lame!
The address of this page is: http://www.amiright.com/parody/90s/xuxa2.shtml For help, see the examples of how to link to this page.