In the New Field of "Neurotheology," Scientists Seek the Biological Basis of Spirituality. Is God All in Our Heads?
Austin suddenly felt a sense of enlightenment unlike anything he had ever experienced. His sense of individual existence, of separateness from the physical world around him, evaporated like morning mist in a bright dawn. He saw things "as they really are," he recalls. The sense of "I, me, mine" disappeared. "Time was not present," he says. "I had a sense of eternity. My old yearnings, loathings, fear of death and insinuations of selfhood vanished. I had been graced by a comprehension of the ultimate nature of things." Call it a mystical experience, a spiritual moment, even a religious epiphany, if you like--but Austin will not. Rather than interpret his instant of grace as proof of a reality beyond the comprehension of our senses, much less as proof of a deity, Austin took it as "proof of the existence of the brain." He isn't being smart-alecky. As a neurologist, he accepts that all we see, hear, feel and think is mediated or created by the brain. Austin's moment in the Underground therefore inspired him to explore the neurological underpinnings of spiritual and mystical experience. In order to feel that time, fear and self-consciousness have dissolved, he reasoned, certain brain circuits must be interrupted. Which ones? Activity in the amygdala, which monitors the environment for threats and registers fear, must be damped. Parietal-lobe circuits, which orient you in space and mark the sharp distinction between self and world, must go quiet. Frontal- and temporal-lobe circuits, which mark time and generate self-awareness, must disengage. When that happens, Austin concludes in a recent paper, "what we think of as our 'higher' functions of selfhood appear briefly to 'drop out,' 'dissolve,' or be 'deleted from consciousness'." When he spun out his theories in 1998, in the 844-page "Zen and the Brain," it was published not by some flaky New Age outfit but by MIT Press.
What all the new research shares is a passion for uncovering the neurological underpinnings of spiritual and mystical experiences-for discovering, in short, what happens in our brains when we sense that we "have encountered a reality different from--and, in some crucial sense, higher than--the reality of every- day experience," as psychologist David Wulff of Wheaton College in Massachusetts puts it. In neurotheology, psychologists and neurologists try to pinpoint which regions turn on, and which turn off, during experiences that seem to exist outside time and space. In this way it differs from the rudimentary research of the 1950s and 1960s that found, yeah, brain waves change when you meditate. But that research was silent on why brain waves change, or which specific regions in the brain lie behind the change. Neuro-imaging of a living, working brain simply didn't exist back then. In contrast, today's studies try to identify the brain circuits that surge with activity when we think we have encountered the divine, and when we feel transported by intense prayer, an uplifting ritual or sacred music. Although the field is brand new and the answers only tentative, one thing is clear. Spiritual experiences are so consistent across cultures, across time and across faiths, says Wulff, that it "suggest[s] a common core that is likely a reflection of structures and processes in the human brain."
There was a feeling of energy centered within me... going out to infinite space and returning. There was a relaxing of the dualistic mind, and an intense feeling of love. I felt a profound letting go of the boundaries around me, and a connection with some kind of energy and state of being that had a quality of clarity, transparency and joy. I felt a deep and profound sense of connection to everything, recognizing that there never was a true separation at all.
In a typical run, Baime settled onto the floor of a small darkened room, lit only by a few candles and filled with jasmine incense. A string of twine lay beside him. Concentrating on a mental image, he focused and focused, quieting his conscious mind (he told the scientists afterward) until something he identifies as his true inner self emerged. It felt "timeless and infinite," Baime said afterward," a part of everyone and everything in existence." When he reached the "peak" of spiritual intensity, he tugged on the twine. Newberg, huddled outside the room and holding the other end, felt the pull and quickly injected a radioactive tracer into an IV line that ran into Baime's left arm. After a few moments, he whisked Baime off to a SPECT (single photon emission computed tomography) machine. By detecting the tracer, it tracks blood flow in the brain. Blood flow correlates with neuronal activity.
I felt communion, peace, openness to experience... [There was] an awareness and responsiveness to God's presence around me, and a feeling of centering, quieting, nothingness, [as well as] moments of fullness of the presence of God. [God was] permeating my being.
I could hear the singing of the planets, and wave after wave of light washed over me. But... I was the light as well... I no longer existed as a separate 'I'... I saw into the structure of the universe. I had the impression of knowing beyond knowledge and being given glimpses into ALL.
I was alone upon the seashore... I felt that I... return[ed] from the solitude of individuation into the consciousness of unity with all that is... Earth, heaven, and sea resounded as in one vast world encircling harmony... I felt myself one with them.


text:
The Revolutionary Tech Breathing New Life Into A $600 Billion Industry
FN Media Group Presents Oilprice.com Market Commentary
London – August 13, 2020 – Combine a Super Bowl-winning quarterback with some of the biggest names in the tech industry and you could be looking at the phenomenally lucrative future of sports. Only this time, the biggest winner may be a small Canadian startup that's got their attention. Mentioned in today's commentary includes: The Walt Disney Company (NYSE: DIS), Electronic Arts Inc. (NASDAQ: EA), Microsoft Corporation (NASDAQ: MSFT), Amazon.com, Inc. (NASDAQ: AMZN), BlackRock, Inc. (NYSE: BLK).
Facedrive (FD.V; FDVRF), a full-scale ESG platform, has already disrupted every tech-heavy segment from ride-sharing and food delivery to COVID contact tracing. Now, it's preparing to jump into the sports arena – and it's got some major firepower behind it.
That firepower includes NFL superstar Russell Wilson and investment vehicles belonging to some of the big names in tech. Or, rather—the biggest names in tech: Amazon CEO Jeff Bezos, Alibaba co-founder Joseph Tsai, YouTube founder Chad Hurley.
Here's how it's happening: Facedrive is acquiring an equity interest in a sports celebrity-studded company called Tally Technologies, which is looking revolutionize engagement between major sports leagues, the teams and their fans.
Tally evolved out of TraceMe, a celebrity-studded content app founded by Wilson with early-in investors from the biggest tech companies in the world. That early funding came from Bezos Expeditions, the Amazon CEO's investment vehicle, and from two other investments vehicles attached to the Alibaba and YouTube founders. TraceMe was acquired by Nike last year, but Wilson hedged his bets that Tally would be huge, so he held onto it and then went looking for the right partner to build it out. He could have chosen anyone, but he chose Facedrive, which puts the Canadian ESG tech platform on the radar of some very big names.
Now, because the transaction is a share exchange that will see Facedrive inject $1M in cash in Tally and $2M in Facedrive shares, the biggest names in the tech industry will indirectly become shareholders in one of Canada's most promising ESG tech platforms.
Overnight, it's a new business for Facedrive on multiple fronts. And it makes sense for Tally, too: Tally sees a huge opportunity for a massive gamification platform by partnering with Facedrive, and with Facedrive CEO Sayyan Navratnam on the board, they'll be getting access to lots of new technology and expertise, as well as the potential to bring on new teams and partners.
For Facedrive, it's a major new venture that serves as confirmation that it's tech ecosystem not only works, but it works for all sorts of industries.
For investors interested in both sustainability and the extraordinary growth potential of tech stocks, Facedrive is now a one stop shop for the hottest sectors in this space. And with some of the biggest names coming on board as shareholders through their investment vehicles, the sky could be the limit for this burgeoning startup.
Why Soaring Tech Is the Future of Sports
The tech-heavy NASDAQ just logged its 30th record of the year … and tech stocks are absolutely killing it. That's because tech has emerged as the future of pretty much every single industry on the planet.
That's what Facedrive has been saying since 2016, and why it's developed an entire tech ecosystem that has tentacles in a huge lineup of industry segments. Now, it's adding sports predictions to the roster in a very big way (think: NFL, NBA, NHL, MLB). And for the sports industry in the middle of a pandemic, tech could be the element that makes or breaks it.
In fact, an entire industry has emerged of its own – the "sports tech industry". Disney (DIS), for example, isn't necessarily a brand that many associate with the sports world. It is mostly known for its film and television dynasty. But thanks to its acquisition of ESPN in February 1996, it has quietly grown to become the de facto leader in sports broadcasting. This status was really drilled into place in 2019 when Disney acquired the Fox network, including Fox's massive sports umbrella.
In addition to its massive media empire, however, Disney also opened up its ESPN Wide World of Sports Complex at the Disney World Resort in Lake Buena Vista, Florida to major league sports including the NBA and MLS. Though the COVID-19 crisis had pressed pause on major league sports, Disney is helping fans get their fix through its secure and clean sports stadiums.
Electronic Arts (EA) is another company that's been able to see positive gains and retain its customers even as traditional sports leagues were forced to postpone their seasons. That's largely thanks to its EA Sports brand. EA has been at the forefront of the sports video game niche since the 1990s, securing deals with every major sports league to utilize the team names, players and even iconic stadiums to help fans truly emerge themselves in that world.
Fans can participate in practically any sport thanks to the company's numerous titles, including the FIFA, Madden NFL, UFC, MLB, NHL, NBA, and even PGA brands. And within each video game, fans can rediscover their favorite teams and players in an entirely new way. The games often even allow gamers to manage their teams and engage in the 'business' side of things.
The game has changed, and Facedrive and Tally are in a position to pioneer what could prove to be a very popular sports app. Facedrive (FD.V; FDVRF) is essentially acquiring an interest in the future of sports, and it works like this:
Tally is a new tech platform that seeks to do what sports can no longer do on its own: deepen engagement around live events. With eSports already working to "democratize" major sporting events and make them accessible to a massive global audience, Tally is completely changing the game in a different way.
Harnessing the Real Revenue Potential
Gamification and online fan engagement are the key to sports revenue now. And Tally is the premier tech solution for increasing audience engagement during live sporting events.
Tally's technology enables sports teams, venues, and broadcasters to dramatically increase viewer engagement, which has been culled by a pandemic, create new revenue opportunities, and build direct relationships with new and existing customers.
Tally offers a complete, state-of-the-art solution supporting the creation and delivery of partner-branded, free-to-play prediction games for sports fans all around the world.
Robust game authoring and validation
A mobile-first app for fans
Comprehensive analysts for partners
A modular system with a wide range of game formats that is easy to integrate with other apps and eBay to access from multiple devices
Built-in monetization tools with everything from leaderboard prizes, social shares, survey integration, and exclusive sponsor shout-outs to giveaways
And for the fans, that predictive experience is the key to wildly deeper engagement. Via the mobile app, fans can engage on multiple, free-to-play predictive levels.
The Tally tech platform is a natural extension to gaming and sports betting, making the growth prospects of this new Facedrive/Tally tie-up even more alluring.
Tally enables large scale 'streak' prediction prizing and can mirror traditional game by game Player props, Team props, Intermission score predictions, and Over/Under prop predictions to educate the new user to sports betting. It will activate new and dormant sports betting accounts directly from the platform driving unrivaled lead generation conversion.
And, of course, it helps that it was founded by one of the biggest names in American football–Super Bowl-winning quarterback Russell Wilson, and shot through with investor confidence by everyone from Bezos' investment vehicle to YouTube and Alibaba, all big names who initially invested with Wilson.
The 'Ecosystem' Blitzkrieg
If you missed Facedrive's (FD.V; FDVRF) debut of its massive ESG platform, that's no matter. The Tally acquisition could potentially be the biggest move this company has made so far, even in a string of rapid-fire acquisitions and partnerships.
When Facedrive publicly launched in 2019 with its initial ride-hailing platform that was the first to offer EVs and a carbon-neutral lift, that was simply the first salvo. Then came long-distance car-pooling to corner every mile with its acquisition of HiRide.
Then, in rapid succession, it launched its food delivery platform–Facedrive Foods–and acquired the Foodora Canada assets of wildly successful international giant Delivery Hero.
And since then, the news flow hasn't let up. It immediately established itself as one of the leaders in Canada in the COVID contact-tracing field, with the launch of TraceScan in partnership with one of the world's biggest labor unions (guaranteeing immediate deployment of this unique tech and its convenient wearables) and one of Canada's top tech institutions.
Its social distance and entertainment app, HiQ, raced to the one million downloads mark just five weeks after launching. Now it is one of the top 5 trivia apps in over 115 countries. And, now, it's jumped into the sports tech industry in one of its biggest moves yet, for a touchdown in the most lucrative arena – all with a premium that links back to the biggest names in the global tech industry. And it's only natural that it is following in the footsteps of tech giants.
Microsoft (MSFT) is a company that has heavily invested in building its own ecosystem, including in the sports realm. Microsoft has partnerships with the NHL, the PGA, NASCAR and even specific teams like Real Madrid and the Renault Sports Formula One team. Microsoft has built custom platforms for these enterprises using its Microsoft Surface technology. The tech helps reimagine and create entirely new strategies, giving teams the edge they need to stay ahead of the competition.
Amazon (AMZN) is another giant company with an entire ecosystem behind it. From retail to cloud services utilized by companies across the world, it's made all the right moves to secure its spot in the trillion-dollar club. And it hasn't ignored the realm of sports, either. Through its Twitch acquisition, Amazon has been able to essentially corner the esports market. And the esports boom has turned the heads of many in the traditional sports industry, as well. Already, Twitch has received the go-ahead to cover a number of major league audiences, including those from the NBA, the Premier League, the UFC and more.
These days, it's almost hard to not mention BlackRock (BLK) when talking about any new investment trend. BlackRock is already the de facto leader in the environment, sustainability, and governance (ESG) trend, and it's also a major investor in just about everything technology-related. But those are only a portion of its investments. BlackRock is also involved in the sports world in a big way. As the primary stakeholder in the iconic Sports Illustrated magazine, it's impossible not to suggest that the investment giant will not benefit from the looming comeback in traditional sports.
By. Caroline Davey
**IMPORTANT! BY READING OUR CONTENT YOU EXPLICITLY AGREE TO THE FOLLOWING. PLEASE READ CAREFULLY**
This publication contains forward-looking information which is subject to a variety of risks and uncertainties and other factors that could cause actual events or results to differ from those projected in the forward-looking statements. Forward looking statements in this publication include that the demand for ride sharing services will grow; that the Tally app will become popular and start generating substantial revenues; that the Tally app will fit with Facedrive's other brands; that the sports predictive app will lead to online sports betting; that Facedrive will be able to fund its capital requirements in the near term and long term; and that Facedrive will be able to carry out its business plans. These forward-looking statements are subject to a variety of risks and uncertainties and other factors that could cause actual events or results to differ materially from those projected in the forward-looking information. Risks that could change or prevent these statements from coming to fruition include that the Tally app may not become popular, may not lead to revenues from gaming and that competitors may offer better or cheaper alternatives; changing governmental laws and policies; the company's ability to obtain and retain necessary licensing in each geographical area in which it operates; the success of the company's expansion activities and whether markets justify additional expansion; the ability of the company to attract a sufficient number of drivers to meet the demands of customer riders; the ability of the company to attract drivers who have electric vehicles and hybrid cars; and the ability of Facedrive to attract providers of good and services for partnerships on terms acceptable to both parties, and on profitable terms for Facedrive; that the products co-branded by Facedrive may not be as merchantable as expected. The forward-looking information contained herein is given as of the date hereof and we assume no responsibility to update or revise such information to reflect new events or circumstances, except as required by law.
ADVERTISEMENT. This communication is not a recommendation to buy or sell securities. An affiliated company of Oilprice.com, Advanced Media Solutions Ltd, and their owners, managers, employees, and assigns (collectively "the Company") has signed an agreement to be paid in shares to provide services to expand ridership and attract drivers in certain jurisdictions. In addition, the owner of Oilprice.com has acquired a considerable number of additional shares of FaceDrive (FD.V) for personal investment and is negotiating to acquire more. This compensation and share acquisition resulting in the beneficial owner of the Company having a major share position in FD.V is a major conflict with our ability to be unbiased, more specifically:
This communication is for entertainment purposes only. Never invest purely based on our communication. Therefore, this communication should be viewed as a commercial advertisement only. We have not investigated the background of the featured company. Frequently companies profiled in our alerts experience a large increase in volume and share price during the course of investor awareness marketing, which often end as soon as the investor awareness marketing ceases. The information in our communications and on our website has not been independently verified and is not guaranteed to be correct.
SHARE OWNERSHIP. The owner of Oilprice.com owns shares of this featured company and therefore has a substantial incentive to see the featured company's stock perform well. The owner of Oilprice.com will not notify the market when it decides to buy more or sell shares of this issuer in the market. The owner of Oilprice.com will be buying and selling shares of this issuer for its own profit. This is why we stress that you conduct extensive due diligence as well as seek the advice of your financial advisor or a registered broker-dealer before investing in any securities.
NOT AN INVESTMENT ADVISOR. The Company is not registered or licensed by any governing body in any jurisdiction to give investing advice or provide investment recommendation. ALWAYS DO YOUR OWN RESEARCH and consult with a licensed investment professional before making an investment. This communication should not be used as a basis for making any investment.
RISK OF INVESTING. Investing is inherently risky. Don't trade with money you can't afford to lose. This is neither a solicitation nor an offer to Buy/Sell securities. No representation is being made that any stock acquisition will or is likely to achieve profits.
DISCLAIMER: OilPrice.com is Source of all content listed above. FN Media Group, LLC (FNM), is a third party publisher and news dissemination service provider, which disseminates electronic information through multiple online media channels. FNM is NOT affiliated in any manner with OilPrice.com or any company mentioned herein. The commentary, views and opinions expressed in this release by OilPrice.com are solely those of OilPrice.com and are not shared by and do not reflect in any manner the views or opinions of FNM. FNM is not liable for any investment decisions by its readers or subscribers. FNM and its affiliated companies are a news dissemination and financial marketing solutions provider and are NOT a registered broker/dealer/analyst/adviser, holds no investment licenses and may NOT sell, offer to sell or offer to buy any security. FNM was not compensated by any public company mentioned herein to disseminate this press release.
FNM HOLDS NO SHARES OF ANY COMPANY NAMED IN THIS RELEASE.
This release contains "forward-looking statements" within the meaning of Section 27A of the Securities Act of 1933, as amended, and Section 21E the Securities Exchange Act of 1934, as amended and such forward-looking statements are made pursuant to the safe harbor provisions of the Private Securities Litigation Reform Act of 1995. "Forward-looking statements" describe future expectations, plans, results, or strategies and are generally preceded by words such as "may", "future", "plan" or "planned", "will" or "should", "expected," "anticipates", "draft", "eventually" or "projected". You are cautioned that such statements are subject to a multitude of risks and uncertainties that could cause future circumstances, events, or results to differ materially from those projected in the forward-looking statements, including the risks that actual results may differ materially from those projected in the forward-looking statements as a result of various factors, and other risks identified in a company's annual report on Form 10-K or 10-KSB and other filings made by such company with the Securities and Exchange Commission. You should consider these factors in evaluating the forward-looking statements included herein, and not place undue reliance on such statements. The forward-looking statements in this release are made as of the date hereof and FNM undertakes no obligation to update such statements.
Media Contact e-mail: editor@financialnewsmedia.com U.S. Phone: +1(954)345-0611
SOURCE: Oilprice.com
RECENT FNM NEWS COVERAGE
Vitally Important COVID-19 Test Kit Market Could Exceed $15 Billion This Year Alone
Economic Uncertainty Driving Gold Prices To Record Highs
Massive Consolidations Taking Place Across Health Care, Especially Among Private Practices
Fluorspar Mining Efforts Picking Up Pace as Steady Demand for Fluorite Material Rises
Global Electrophysiology Devices Market Could Exceed $12.2 Billion By 2026
Sign Up & Get FREE News Alerts From FNM Today!
Terms of Service | Privacy Policy | Disclaimer
FinancialNewsMedia.com (FNM) is a third party publisher and news dissemination solutions provider of multimedia platforms that enable officers & directors, corporate communicators, public relations and investor relations officers to leverage content to engage with all their key audiences. Any ideas and opinions presented in any FNM News Alerts, Video clips, Press Releases, RSS Feeds or Social Network Posts are for informational purposes only, and do not reflect the opinions of FNM or any of its affiliates, subsidiaries or partners. UNDER NO CIRCUMSTANCES SHOULD ANY CONTENT CONTAINED HEREIN BE INTERPRETED TO REPRESENT TRADING OR INVESTMENT ADVICE. All viewers agree that under no circumstances will FNM, its subsidiaries, partners, officers, employees, affiliates, or agents be held liable for any loss or damage caused by your reliance on information contained on this or any affiliated website.
© 2020 Financial News Media. All Rights Reserved.

redpajama_set_name: RedPajamaCommonCrawl


text:
April 13, 2018 / Outdoor Play, Sydney, Sydney Life / 0 comment
Sydney Harbour is such a showoff. Sparkling blue water with white cut cliffs, pristine beaches and fascinating history to boot. It's easy to see the best Sydney has to offer by simply heading out for a day trip on the Sydney Harbour ferries.
1 7 Awesome Days Trips On Sydney Harbour Ferries
1.1 Cockatoo Island
1.2 Taronga Zoo
1.3 Watson's Bay
1.4 Manly Beach
1.5 Luna Park, Kirribilli
1.6 Darling Harbour
1.7 Parramatta
1.8 More things to do in Sydney
7 Awesome Days Trips On Sydney Harbour Ferries
Cockatoo Island
The largest island on Sydney Harbour is also a fascinating historical site. Cockatoo Island is a UNESCO World Heritage Site that includes the only remaining dry dock in Australia built using convict labour. Before the First Fleet arrived and used convict labour to turn the island into a prison and then shipbuilding dock, the island was a base for the local Aboriginal tribe, the Eora people, to fish from and built canoes.
After a significant restoration, Cockatoo Island has been opened to the public and makes for an incredible day trip exploring the convict relics, industrial buildings and historic maritime dock.
The island also plays host to events and exhibitions such as the Biennale of Sydney, Australia's largest contemporary visual arts event. Open until June 11, 2018, the exhibition is held in multiple venues across Sydney. On Cockatoo Island the artworks reflect the history of the site around themes such as movement, migration, production and participation. The artworks, by 20 artists from all over the world, are designed to evolve over the course of the Biennale.
The Biennale of Sydney is brought to Cockatoo Island this year with the support of principle partner TWT Property Group, whose mission is to enrich the communities they create with art.
You can read more about a day trip to Cockatoo island here.
Catch the ferry directly to Taronga Zoo for a walk on the wild side. A non-profit organisation supporting wildlife conservation, a visit to Taronga is as educational as it is entertaining. There are over 4,000 animals to meet at the zoo, including Australian native wildlife and rare and exotic animals.
The QBE Free-Flight Bird Show is a highlight for us, to witness spectacular birds, most of whom have been rescued and rehabilitated, soar across the sky with the Sydney Harbour as an impressive backdrop.
Kids will love the two playgrounds, water features, up-close animal experiences such as the lemur walk-in enclosure and the new Wile Ropes adventures through the treetops.
Get more info on a day at Taronga Zoo with kids here.
A short 20 minutes ferry trip from Circular Quay will take you to Australia's oldest fishing village, Watson's Bay. This sparkling gem of a spot is home to the iconic seafood institution Doyles, perfect for grabbing takeaway fish and chips or, for a more upmarket dining experience, try the celebrated restaurant Doyle's on the Beach.
We love Robertson Park, the large, leafy park behind the beach, for a picnic or a run around the excellent playground.
Don't forget to pack the swimmers as Watson's Bay is excellent for a splash on the beach with the gorgeous city view behind you.
Get more info on a day trip to Watson's Bay here.
A 30 minute ferry trip will take you to Manly, the site of the first world surfing competition in 1964. Manly Wharf has plenty of bars, cafes and restaurants to enjoy, and, if you're lucky, you might even see some of Manly's Little Penguins underneath the wharf.
Stroll down the Corso, a pedestrian mall filled with cafes, surf shops and pubs, to the ocean beach.
At the beach you'll be enticed by the white sand and sparkling water to dive on in. If you want to give surfing a go, local businesses Manly Surf School and Manly Surf Guide can lend a hand.
We love to take the scenic walking path south to Fairy Bower Pool and Shelly Beach, the later of which is the perfect family spot for a swim in calm waters.
Get more info here on a day trip to Manly.
Luna Park, Kirribilli
Catch a Sydney Harbour ferry to Kirribilli and Luna Park, an historical Sydney icon that was listed on the State Heritage Register in 2010. Constructed in its current scenic spot at the foot of the Sydney Harbour Bridge in 1935, Luna Park is an amusement park with the best view in town.
We particularly love Cony Island Funnyland, which is the only operating example of a 1930s funhouse left in the world.
Coney Island was built in 1935, and although there have been some changes made over the years, the layout is almost identical to when it opened, including the rotating barrels, moving platforms, long slides and arcade games that line the walls.
Get more info on visiting Luna Park here.
This entertainment precinct is jam packed with attractions that the whole family will enjoy. Get up close to rescued dugongs and king and gentoo penguins at SEA LIFE Sydney Aquarium, meet a koala and gigantic croc at WILD LIFE Sydney Zoo, or take a selfie with a celebrity at Madame Tussauds, the famous wax museum.
We also love the ever-changing exhibitions at the Australian National Maritime Museum (ANMM), which is particularly kid-friendly and offers plenty of kid-focussed activities during school holidays, and the Chinese Garden of Friendship for a tranquil afternoon.
The Darling Quarter Kids Playground is one of the best in Sydney. The slides, flying fox, swings, climbing ropes and splash area are not to be missed.
Get more info on visiting SEA LIFE Sydney Aquarium, the Penguin Expedition at SEA LIFE Sydney Aquarium, WILD LIFE Sydney Zoo, Madame Tussauds, the ANMM, Chinese Garden of Friendship and Darling Quarter Kids Playground.
Catch a Sydney Harbour ferry along scenic Parramatta River to Parramatta Wharf and enjoy a day exploring Australia's fascinating past and vibrant, culturally-diverse present.
Parramatta was founded in 1788 and is the second-oldest city in Australia, only 10 months younger than Sydney. It's home to two UNESCO World Heritage Australian Convict Sites: the Convict-built Old Government House, which was the country residence of the first 10 Australian governors, and Parramatta Park, where colonists, both convict and free, grew their crops before being gazetted as a People's Park in 1858.
Parramatta Park has two playgrounds, including the exceptional Domain Creek Playground, and an enclosed, formal rose garden featuring a are collection of heritage roses.
Stroll down Church Street for a diverse variety of restaurants and a scoop of innovative gelato at Messina Gelato.
We love nearby Elizabeth Farm, family home of wool pioneer, John and his wife Elizabeth Macarthur, and Australia's oldest European building.
Get more info on visiting Parramatta, the Domain Creek Playground and Elizabeth Farm.
Get more info on catching a Sydney Harbour Ferry here.
More things to do in Sydney
Find a list of the best things to do in Sydney with kids here.
Find things to do in Sydney for free here.
Get a list of the best kid-friendly beaches in Sydney.
Find the best animal experiences in Sydney here.
Visit the best kid-friendly restaurants in Sydney here.
Find the best high teas in Sydney here.
Head to one of the best museums in Sydney here.
Check out things to do in Sydney on a rainy day here.
Enjoy one of the best ocean pools in Sydney here.
First time visiting Sydney? Get tips for first time visitors to Sydney here.
This post is a collaboration with TWT Property. All opinions are my own.
SEA LIFE Sydney Aquarium Darling Harbour Australia
Balmoral Beach Sydney: Sydney's Best Beaches
Chinese Garden of Friendship, Darling Harbour,…
Novotel Darling Harbour: Sydney Hotel Review
« Travel Hacks: Top Tips For Flying With Kids
Sydney Day Trips: The Australian Reptile Park »


text:
The Billionaire Space Race Is a Wake-up Call
July 22nd, 2021 - by Sam Pizzigati / Other Words
Bored billionaires want tax breaks to
escape a planet they helped pollute.
Should we even let them back?
Sam Pizzigati / Other Words
(July 21, 2021) — Three of the richest billionaires on Earth are now spending billions to exit Earth's atmosphere and enter into space. The world is watching — and reflecting.
Some charmed commentators say the billionaires racing into space aren't just thrilling humankind — they're uplifting us. The technologies they develop "could benefit people worldwide far into the future,"says Yahoo Finance's Daniel Howley.
But most commentators seem to be taking a considerably more skeptical perspective.
They're dismissing the space antics of Branson, Bezos, and Musk as the ego trips of bored billionaires — "cynical stunts by disgustingly rich businessmen," as one British analyst puts it, "to boost their self-importance at a time when money and resources are desperately needed elsewhere."
"Space travel used to be about 'us,' a collective effort by the country to reach beyond previously unreachable limits," writes author William Rivers Pitt. "Now, it's about 'them,' the 0.1 percent."
The best of these skeptical commentators can even make us laugh.
"Really, billionaires?" comedian Seth Meyers asked earlier this month. "This is what you're going to do with your unprecedented fortunes and influence? Drag race to outer space?"
Let's enjoy the ridicule. But let's not treat the billionaire space race as a laughing matter.
Let's see it as a wake-up call — a reminder that we don't only get billionaires when wealth concentrates. We get a society that revolves around the egos of the most affluent and an economy where the needs of average people don't particularly matter.
Characters like Elon Musk, notes Paris Max, host of the Tech Won't Save Us podcast, are using "misleading narratives about space to fuel public excitement" and gain tax-dollar support for various projects "designed to work best — if not exclusively — for the elite."
The three corporate space shells for Musk, Bezos, and Branson — SpaceX, Blue Origin, and Virgin Galactic — have "all benefited greatly through partnerships with NASA and the US military," notes CNN Business. Their common corporate goal: to get satellites, people, and cargo "into space cheaper and quicker than has been possible in decades past."
Branson is hawking tickets for roundtrips "to the edge of the atmosphere and back" at $250,000 per head. He's planning some 400 such trips a year, observes British journalist Oliver Bullough, about "almost as bad an idea as racing to see who can burn the rainforest quickest."
The annual UN Emissions Gap Report last year concluded that the world's richest 1 percent do more to foul the atmosphere than the entire poorest 50 percent combined. Opening space to rich people's joyrides would stomp that footprint even bigger.
Bezos and Musk seem to have grander dreams than mere space tourism — they're looking to colonize space. They see space as a refuge from an increasingly inhospitable planet Earth. And they expect tax-dollar support to make their various pipedreams come true.
How should we respond to all this?
We should, of course, be working to create a more hospitable planet for all humanity. In the meantime, advocates are circulating tongue-in-cheek petitions that urge terrestrial authorities not to let orbiting billionaires back on Earth.
"Billionaires should not exist…on Earth or in space, but should they decide the latter, they should stay there," reads one Change.org petition nearing 200,000 signatures.
Ric Geiger, the 31-year-old automotive supplies account manager behind that effort, is hoping his petition helps the issue of maldistributed wealth "reach a broader platform."
Activists like Geiger are going down the right track. We don't need billionaires to "conquer space." We need to conquer inequality.
Billionaires Won't Save the World — Just Look at Elon Musk
(March 21, 2018) — Will Mars save humanity? Or will our savior be billionaire Elon Musk?
Musk, the CEO of SpaceX and Tesla, humbly believes we don't have to choose. Mars will save us, he promises, and Musk himself will engineer this Mars miracle.
In 2019, Musk claims, SpaceX will start making short trips to Mars. By the early 2020s, his company will begin colonizing the Red Planet with a human population.
Why this feverish haste to set foot on interplanetary terra firma?
Musk sees a new "dark age" descending on our precious Earth. Another world war — or some environmental collapse — appears likely to threaten us with extinction, he fears.
Mars strikes Musk as our ideal refuge, the place where humankind will heroically regroup and eventually "bring human civilization back" to our mother planet.
And we can even have some fun in the process. The Mars colony that Musk envisions will have everything from iron foundries to "pizza joints and nightclubs."
"Mars," he quips, "should really have great bars."
Reporters have become accustomed to this sort of visionary whimsy from Musk. The billionaire, In These Times says, has crafted his image as "a quirky and slightly off-kilter playboy genius inventor capable of conquering everything from outer space to the climate crisis with the sheer force of his imagination."
This carefully cultivated image has proven extraordinarily lucrative.
Investors now value Tesla, his 15-year-old car company, at around $60 billion — not bad, note Wall Street watchdogs Pam and Russ Martens, for a firm that "lost almost $2 billion last year and has never delivered an annual profit to shareholders."
But Musk remains supremely confident that his enterprise on Mars will take root and prosper. He's betting a good chunk of his fortune on that.
Or rather, he's betting a good chunk of taxpayers' fortune.
Musk owes his billions, as commentator Kate Aronoff points out, to the billions in direct taxpayer subsidies his companies have received over the years — and the billions more in taxpayer-funded research into rocket technology and other high-tech fields of knowledge.
So Musk is essentially investing our billions in his own pet projects, everything from the Mars gambit to establishing a mass-market niche for high-tech flamethrowers.
None of this is going to rescue humanity anytime soon.
Indeed, if Musk really wanted to ensure humankind a sustainable future, he wouldn't be plotting escapes to Mars or marketing flamethrowers to the masses. He'd be challenging the global economic status quo that's left him phenomenally rich and our world phenomenally unequal.
This inequality may well pose the greatest threat to our well-being as a species. Stark economic divides invite armed confrontations.
Inequality and conflict, Norwegian scholars observed last year in a major report for the United Nations and the World Bank, remain "inextricably linked." They found that "inequality influences the outbreak and dynamics of violent conflict," going all the way back to the ancient Greeks.
In more recent years, researchers have made great strides in understanding the actual pathways in unequal societies that turn conflict violent. But huge gaps in the research are still frustrating our understanding.
What we do know: Hawking high-tech flamethrowers is never going to save humanity. Neither will bar-hopping on Mars.


text:
Vox Balaenae (Voice of the Whale)
Yesterday, we went to Judi Lampert's faculty flute recital at UNCA. Now, I'm not a huge fan of the flute; without my hearing aid, I can't hear it in its higher ranges, and with the hearing aid, the shrillness borders on—and sometimes surpasses—an experience of acute pain. I probably wouldn't have gone (no offense to the flautists of the world), except that my piano teacher, Deborah Belcher, was going to be playing in the trio performance of American composer George Crumb's Vox Balaenae (Voice of the Whale).
The concert was a performance of exclusively 20th-century music. Now, I don't have a great deal of experience listening to such. LSU always had their Festival of Contemporary Music, and I attended many of the performances there when I lived in Baton Rouge. And I've seen a few symphony orchestras perform Shostakovich, Stravinsky, and Copland … but I've never really known enough about the music to get a good sense of it, or, I'm afraid, to develop a true appreciation for it. Add to that the fact that I know very few "classical music" listeners who seriously listen to compositions written after the 19th century (I have been guilty of that from time to time), and I haven't exactly built an grand appreciation for the atonal styles and the weird rhythms found in much 20th-century music.
I do occasionally listen to recordings of 20th-century compositions on naxos.com. Sometimes I find a composer I really like, and other times, I'm left wishing that someone could explain to me why a certain piece is considered "music." I really want to know. I just can't figure it all out on my own.
Anyway, I've gotten off the subject. Back to Vox Balaenae.
Here is a description of the piece, in Crumb's own words:
Vox Balaenae (Voice of the Whale), composed in 1971 for the New York Camerata, is scored for flute, cello and piano (all amplified in concert performance). The work was inspired by the singing of the humpback whale, a tape recording of which I had heard two or three years previously. Each of the three performers is required to wear a black half-mask (or visor-mask). The masks, by effacing the sense of human projection, are intended to represent, symbolically, the powerful impersonal forces of nature (i.e. nature dehumanized). I have also suggested that the work be performed under deep-blue stage lighting.
The form of Voice of the Whale is a simple three-part design, consisting of a prologue, a set of variations named after the geological eras, and an epilogue.
The opening Vocalise (marked in the score: "wildly fantastic, grotesque") is a kind of cadenza for the flutist, who simultaneously plays his instrument and sings into it. This combination of instrumental and vocal sound produces an eerie, surreal timbre, not unlike the sounds of the humpback whale. The conclusion of the cadenza is announced by a parody of the opening measures of Strauss' Also sprach Zarathustra.
The Sea-Theme ("solemn, with calm majesty") is presented by the cello (in harmonics), accompanied by dark, fateful chords of strummed piano strings. The following sequence of variations begins with the haunting sea-gull cries of the Archezoic ("timeless, inchoate") and, gradually increasing in intensity, reaches a strident climax in the Cenozoic ("dramatic, with a feeling of destiny"). The emergence of man in the Cenozoic era is symbolized by a partial restatement of the Zarathustra reference.
The concluding Sea-Nocturne ("serene, pure, transfigured") is an elaboration of the Sea-Theme. The piece is couched in the "luminous" tonality of B major and there are shimmering sounds of antique cymbals (played alternately by the cellist and flutist). In composing the Sea-Nocturne I wanted to suggest "a larger rhythm of nature" and a sense of suspension in time. The concluding gesture of the work is a gradually dying series of repetitions of a 10-note figure. In concert performance, the last figure is to be played "in pantomime" (to suggest a diminuendo beyond the threshold of hearing!); for recorded performances, the figure is played as a "fade-out".
The only disappointment in yesterday's performance was the blue spotlight (or lack thereof). I think there was a blue spotlight; the piano seemed to have a tinge of blue at certain angles, but that may have just been my imagination wanting to see the blue spotlight. The stage, with its vinyl, wooden-legged chairs, looked less like an undersea world and more like … well, a university's auditorium stage. It didn't help that the auditorium itself wasn't particularly dark, so that took away from the intended mystique of the performance.
That said (and I did say that it was the only disappointment, for me at least), the performance rocked. Well, perhaps "rocked" isn't the right word. It swam. It breathed. It sang. It hummed. It captured the essence, as far as I could tell (with my admitted ignorance regarding atonal music), of the composition. In addition, the artists held my attention for the entire performance. Now, I'm a good listener and I love music concerts, but like most people (I believe), I occasionally find my mind wandering somewhere toward the end of the second movement, and then in the middle of the third movement during a concert. Didn't happen this time. Yes, it was interesting to watch, certainly, but it was the effect of the performance that held my attention.
And if it had that effect on me, then that means they made it accessible. Now, I HATE to use the word "accessible" because it's often synonymous with "dumbed-down." But I don't mean it in that sense. Perhaps the word I am looking for is "listenable." (?) The music certainly wasn't "dumbed down" or "soft," but at the same time it was very ac ... ac ... accessible.
Now, for the performers. They wore, in accordance with the composer's instructions, black half-masks. Deborah looked especially "so cute!" (to use my husband's term), with her glasses worn over the mask.
Judi Lampert started with the Vocalise (the flute "kind of cadenza"), and her singing into the flute did indeed call to mind the eerie but beautiful sound of the humpback whale. It was so "odd" (to use Dan's term again) so see a flute being played, but not to hear "flute sounds" coming out of it. It was even more fascinating to hear whale sounds coming out of it. Very intriguing.
Next, Frances Duff played the cello Sea Theme. Once again, there was the experience of an instrument not making its expected sound. She ran the bow across the strings, but instead of producing the rich tenor voice of a cello, it made high, longing noises … the sounds of sea-gulls calling. Wow. I stared at her bow with a combination of consternation and wonder--the way you'd want to stare at an especially grotesque but strangely beautiful insect.
And then there was Deborah, moving from the piano keys, to the innards of the piano, where she ran a glass rod over the strings to create an "in the depths of the ocean" effect, or lodged a paper clip or chisel in the strings to produce non-piano sounds when she struck the keys. It was amazing to watch. She kept busy, sitting and playing in her graceful style, then standing to produce some strange sound from the inside of the grand. I was so proud of her.
The final section, "Sea Nocturne (…for the end of time), was absolutely beautiful and dreamy. The medium-sized audience pretty much went wild when it was over (if a medium-sized audience can go wild). OK, so we didn't go wild. But we clapped for a long time and the trio had to come out for a second bow. I heard several appreciative hoots.
Was it a good performance? I think so, but, since I have no previous knowledge of experience to which to compare it, I can't say for sure. I found this snippet from a panel discussion with George Crumb at Penn in 2001:
Prof. Narmour, making reference to Beethoven's distance and disinterest from pieces that he had composed earlier in his career, asked Prof. Crumb if a composition composed some thirty years ago maintained any relevance for him as the composer. Crumb's reply was that "with each different group of performers [the Vox Balanae] is reborn" and thus will always maintain a newness that is perhaps obscured by the date of composition. The downside, according to Crumb, is that the music is ever "vulnerable to less than convincing performances."
In my less-than-informed opinion, Judi Lampert, Frances Duff, and Deborah Belcher definitely gave a convincing performance. The music may be vulnerable, but these artists didn't let it down. They didn't let us, the audience, down, either.
Afterward, Deborah said that they had never played the Crumb together so well. The three artists definitely "clicked" up there. No one artist outshone the other, and they just seemed like mouthpieces for the whale- and ocean-like effects that Crumb wanted to produce in his composition.
So maybe the masks had the effect they were intended to have, too.
If anyone is interested, here are the pieces that comprised the concert. The guitar parts were played by Roger Allen Cope.
Histoire du Tango (Bordel 1900) by Astor Piazolla (1921-1992)
Soliloquy by Bertil Van Boer (b. 1924)
Pie'ce by Jacques Ibert (1890-1962)
Assobio a Ja'to (The Jet Whistle) by Heitor Villa Lobos (1881-1945)
Vox Balaenae (Voice of the Whale) for three masked players by George Crumb (b. 1954)
Sonatina by Mario Castelnuovo-Tedesco (1895-1968)
Also (and again, if anyone is interested), here is more information on Vox Balaenae, including the recommended placement of the instruments on the stage and some instances of how the actual music is written on the page.
When I got home last night, I practiced piano for two delicious hours.
Piano/Composition Update
Attention, Jonathan-Knowers
Poetry Tonight!
Well, Lookie Here!
Downloading Music From His Head
Five (Count Em!) Stars!
Anybody Miss Me?
A Mushy Thanksgiving Eve Ramble
Feelin' Moony
Asheville Made the Top 5
I Wanted to Stay Home this Morning
Exciting Things That Happened This Weekend
Of Interest to My WNC Friends
A Navel-Gaze on Writing
Blogging Style?
The Romance-Writing Career that Wasn't
The Grindstone
A Navel-Gaze on Snobbishness
More Boy Scout News ... This Time Not So Good
Nocona Makes It To Canada
Cousin Shorty Has a Blog
The Finer Things in Life
The Importance of Having a Good Daddy
Nocona's Penultimate Day on the Trail
Piano Report
Only in Louisiana!
A Boy Scout Gnu
Happy Birthday, Jack!
D-I-C-T-I-O-N-A-R-Y
Music Update du Jour
G-mail Invites Still Available
I Miss Dr. Greenberg
The More I Learn ...
The Werther's Lady
Too Weird
Feeling Human Again, and Thinking of the Ivory Tower
Blah.
Dizzy Blonde Returns to Work
About the "INFP" in INFPeace
Dizziness Update
Dizzy Blonde
Another Adoptee in the Family
A Little Backyard Hike
Gmail Invites
Fun Friday Humor
Figured Bass
Feeling Sorry for My Kerry Friends
Effuen With Words
Ponder While I Work
Blogging Break
O, Gay Ubya Day!
Someone Said Boo. Yah.
Know Anyone in Nashville?
Return of the Hearing Aid
Kickin' Back ... ?
There's an Essay in the Air
A Mr. Marvin Assignment
A Story of Buts Big and Little
Dear Dubya
Thumb Needs Xanax. So Do the Buds.
Gimme a "Boo!" Gimme a "Hoo!" What's That Spell?
The Triumphant Return of INFPeace


Regional Internet traffic exchange enables and empowers SOX customer with fast and reliable access to other IXPs: VIX in Vienna, RoNIX in Bucharest, Net-IX, B-IX, OM-NIX and T-CIX in Sofia, delivering all the goodies and benefits of remote markets Austrian, Romanian, Hungarian, Bulgarian, Macedonian, Greek, Turkish Internet contents to Serbia, but also distributing Internet content from Serbia to the market bigger than 100 million end Internet users. Regional exchange of Internet traffic started 5 years ago on initiative of our Bulgarian and Romanian colleagues setting the goal of establishing regional IXP over the triangle Bucharest-Sofia-Belgrade. SOX added Vienna to this triangle, and aligned its strategy with old trade route Istanbul-Vienna, also known as Via Militaris dating back to Roman times. Internet giants seems to understand and appreciate this strategy, and made SOX very proud by making it a partner.


text:
\section{Introduction}
Text summarization is the process of automatically generating natural language summaries from an input document while retaining the important points of the text. This process can be helpful for reducing the cost of decision making by reducing the amount of text that needs to be processed, and can aid many downstream applications such as text completion. Summarization can help peoople with hearing disabilites with applications like voice-to-text and also has use-cases like summarizing medical reports for easy access. Additionally, in this day and age of information explosion, precise summarization can help boost human productivity in daily use cases like news, research and context based search rather than keyword searching \cite{quora}. Generating coherent and meaningful summaries of long text is still a big challenge for most machine learning methods. The abstractive summary, unlike extractive summary, is not constrained to reusing the phrases or sentences in the input text and presents major challenges like saliency, fluency and human readability \cite{Shi}. These challenges stem from the fact that the network tries to optimize an objective that does not involve human quality assessment during training, leading to exposure bias. Models trained using a supervised objective exhibit exposure bias \cite{Paulus}, which is a major challenge for fluency in abstractive models. We are introducing the Actor Critic (AC) reinforcement learning (RL) based technique to reward learning appropriate dynamic self attention masks within the language model, which would improve the model's contextual representations. We modify UniLM \cite{UniLM} which is a SoTA model based on BERT that is pre-trained on multiple downstream language generation tasks (Figure \ref{UNI}). The input to our approach is a document and the output is an abstractive summary of the document (Figure \ref{inputoutput}).

\begin{figure*}[htp]
\centering
\includegraphics[height=0.6\textwidth]{UniLM.png}
\caption{UniLM Architecture as presented in \cite{UniLM}}
\label{UNI}
\end{figure*}

\begin{figure*}[htp]
\centering
\includegraphics[scale=0.4]{baseline_diag.png}
\caption{(Left to right) Input Document, Baseline(Extractive Summarization), Oracle}
\label{inputoutput}
\end{figure*}

We use the CNN/Daily Mail dataset \cite{Nallapati}, which consists of online news articles and their corresponding summaries. The breakdown of the dataset is: 287226 (92\%) / 13368 (4\%) / 11490 (4\%) train/val/test examples. The dataset is derived from news articles and their corresponding "highlights" as summaries being the ground truth. An example of the dataset is shown in Figure \ref{inputoutput}. We employ ROUGE metrics to evaluate the summaries. ROUGE \cite{Lin} computes a F1 score by comparing matching sub-phrases (n-grams) in the generated summaries against the ground truth \cite{Rouge}. Our approach has two phases. e finetune the environment (UniLM) for the task of abstractive summarization and also employ the Actor-Critic algorithm to learn the per word importance of the abstractive summarization task through policy gradients. Our hypothesis draws  inspiration from \cite{EASummary} where the authors use extractive summarization as input to an abstractive summarization algorithm to generate coherent summaries. However, instead of feeding sentence level priors we will employ Reinforcement Learning (RL) to inherently learn the importance or redundancy of information contributing to coherent summaries.

\section{Related Work}
Researchers have widely utilized recurrent neural encoder-decoder networks to address the abstractive summarization sequence-to-sequence problem \cite{Shi, Nallapati}. These models are unable to represent long documents or generate multiple coherent sentences. To handle the coherence problem, Wu et al \cite{Wu} introduced a neural coherence model which estimates the degree of coherence between two sentences by their distributed representation. They also introduced a Reinforced Neural Extractive Summarization (RNES) model that utilizes RL to add coherence to a neural extractive summarization.
Lee et al \cite{deepq} estimate Q-values using a neural network that helps maximize ROUGE for Extractive Summarization. The scores are evaluated with the use of word embeddings to capture the similarity between the actual document and the generated summary.
Paulus et al \cite{Paulus} introduced a deep RL method that combines a mixture of supervised sequence-to-sequence modeling to address the abstractive summarization exposure bias introduced by supervised learning. \cite{EASummary}, use a two step approach and employ transformer based models for abstractive summarization. This method uses extractive summarization as a prior to generate abstractive summarization and achieve state-of-the-art (SoTA) ROUGE scores. Generating coherent yet factual summaries has been a challenge for most SoTA approaches where the model traditionally skips over important detail as discussed with examples in \cite{fact}; we aim to address this limitation in abstractive summarization with our approach.
While traditional approaches use predefined self-attention masks or use extractive summarization as a prior to limit redundant information, our approach contains the combined benefits of RL and abstractive summarization to learn the contextual self-attention masks. We hypothesize that this approach will make improvements over the previous methods because the RL objective resembles human quality assessment and abstractive summarization using UniLM generates summaries that are more coherent.

\section{Methods and Architecture}
The presented model studies the effects of using dynamic self-attention masks on the sequence-to-sequence task of abstractive summarization. The model aims to modify the SoTA UniLM and its underlying Transformer architecture to train a reinforcement learning (RL) policy gradient model to generate the self-attention mask. UniLM (UNIfied pre-trained Language Model) contains three types of language modeling tasks: unidirectional, bidirectional, and sequence-to-sequence prediction. UNiLM employs a shared Transformer network and utilizes specific
self-attention masks to control what context the prediction conditions on \cite{UniLM}. Our model specifically tests the use of dynamic self-attention masks on the quality of abstractive summarization. The proposed method trains an Actor Critic (AC) model  [2] to learn a dynamic self-attention distribution over the input tokens with the objective of increasing the summarization output ROUGE score.

\subsection{Actor-Critic Method}
Actor-critic methods are popular deep reinforcement learning algorithms and provide an effective variation of the family of Monte Carlo Policy Gradients (REINFORCE).The policy gradient methods update the probability of stochastic non-deterministic (policy) distribution of actions such that actions with higher expected reward have a higher probability value for an observed state. REINFORCE relies on  estimating policy gradient by sampling using Monte-Carlo methods sample episode and update the policy parameter $\theta$.

\begin{equation}
\nabla_\theta J(\theta) = \mathbb{E}[\sum_{t=0}^{T-1} \nabla_\theta log (\pi_\theta (a_t|s_t)) Q_{\pi_{\theta}}(s_t,a_t)]
\end{equation}

\begin{equation}
\theta = \theta + \alpha_{\theta} \nabla_\theta J(\theta)
\end{equation}

where $Q_{\pi_{\theta}}(s_t,a_t)$ defines the expected $Q$ value .\\

The Actor-Critic (AC) method provides a variation to the REINFORCE algorithm by defining two modules that formalize the two main components in the policy gradient method: policy model and the value function.
\begin{enumerate}
    \item The critic computes value functions to help appraise each action the actor takes. It estimates the $Q_{\pi_{\theta}}(s_t,a_t)$ or $V_{\pi_{\theta}}(s_t)$
    \item The actor is the policy that defines how actions are selected and updates the policy parameters $\theta$ for the policy $\pi_\theta (a_t|s_t)$, in the direction suggested by the critic.

\end{enumerate}

Basic estimation using policy gradients in  Monte Carlo Policy Learning (REINFORCE) and AC methods is inherently noisy. This could lead to unstable convergence and fluctuations in learning \cite{RLBook}. Reward sparsity can also cause unstable learning \cite{AC1}. The Advantage Actor-Critic (A2C) as described in \cite{A2C} provides a solution to high variance policy distribution by subtracting baseline $b(s_t)$ to reduce variance of the policy gradients. The advantage value $A(s_t,a_t) = Q_{\pi_{\theta}}(s_t,a_t) - b(s_t)$. The advantage actor-critic policy gradient:

\begin{equation}
\nabla_\theta J(\theta) = \mathbb{E}[\sum_{t=0}^{T-1} \nabla_\theta log( \pi_{\theta} (a_t|s_t)) A(s_t,a_t)]
\end{equation}

\begin{figure*}[htp]
\centering
\includegraphics[height=0.6\textwidth]{txf.png}
\caption{Encoder with AC Policy Learning for Self-Attention masks}
\label{txf}
\end{figure*}

\subsection{DR.SAS - Attention Mask Policy Learning (A2C)}

Our model modifies the UniLM underlying Transformer encoder to employ the Advantage Actor-Critic method \cite{Konda} for attention mask policy learning. The A2C framework is to train combined Neural Network and Reinforcement Learning approaches by training the A2C Agent to learn suitable policy based off UniLM + ROUGE environment. The definition of our method with regards to the Actor Critic system is as follows:
\begin{itemize}
\item The state space is described as the attention score distribution from an intermediate layer of UniLM for a given input document to be summarized.

\item The Actor learns a policy distribution and the Critic learns the values $Q_t$. Our method maps this policy to the change in attention masks given the state.

\item The policy distribution defines the probabilities of sampling between two values (0 (mask/not attend), 1 (attend)) ($a_t$) to mask a subset of the input sequence $S$ in \ref{UNI}. The Actor-critic policy gradient will help learn the attention mask for each token given the attention distribution. This essentially should help UniLM ignore redundant information in the document to focus on coherent abstractive summarization.

\item Our model utilizes an environment (UniLM + ROUGE reward function) and uses state-based dynamic attention masks to generate the new summary and produce the ROUGE score and correspondingly learns the updated attention score distribution ($s_{t+1}$).

\item  To reduce variance, we utilize the previous value $V_{\pi_{\theta}}(s_t)$ of the attention score distribution state $s_t$ as the baseline function $b(s_t)$. The updated distribution should not have changed much.
\end{itemize}

 The modified encoder with AC Policy Learning for Self-Attention masks is shown in Figure \ref{txf}. We specify in our architecture both the Critic and Actor functions as parameterized neural networks. The Actor-Critic feed forward neural network estimates the value (critic) and the policy (actor) for the continuous action and state spaces. The AC network takes the attention scores (state) of the input tokens and evaluates the state value(s) and policy distribution. The updated policy distribution is then sampled to generate the attention masks. The generated masks are then transformed to values of 0 (attend) and -10000 (mask) to be added to the attention scores that propagate to the UniLM downstream layers to generate the abstractive summary and propagate back the reward (ROUGE) and network gradients.

 The A2C loss is divided into three main components:
 \begin{itemize}
     \item The value (critic) loss:
     \begin{equation}
 \mathcal{L}_{critic} = \frac{1}{2T} \sum_{t=0}^{T-1} A(s_t,a_t)
\end{equation}
\item The policy (actor) loss:
     \begin{equation}
 \mathcal{L}_{actor} = - \frac{1}{2T} \sum_{t=0}^{T-1} log( \pi_{\theta} (a_t|s_t)) A(s_t,a_t)
\end{equation}

\item The entropy loss which measures the dispersion of the policy distribution $P(x)$:
     \begin{align}
         \mathcal{L}_{entropy} &= \mathbb{E}_{x \sim P}[-log(P_t(x))]\\ &= - \frac{1}{2T} \sum_{t=0}^{T-1} \sum_{i=1}^{N=2} -P_t(x_i) log(P_t(x_i))
     \end{align}

 \end{itemize}

The A2C loss is the sum of the three losses:
\begin{equation}
 \mathcal{L}_{a2c} = \mathcal{L}_{actor} + \mathcal{L}_{critic} + \beta \mathcal{L}_{entropy}
\end{equation}

\section{Experimental Results}
In order to evaluate our model, we conducted visual and quantitative analyses. Quantitatively, we evaluate our method in terms of ROUGE. ROUGE stands for Recall-Oriented Understudy for Gisting Evaluation. It is a set of metrics for evaluating automatic summarization of texts as well as machine translation. It works by n-gram comparison between a generated and and reference summary. We compared our model to extractive summarization and the abstractive summarization by UniLM as our baselines. The extractive method selects sentences from a given document to construct a summary of the passage. Specifically, we adapt TextRank \cite{TextRank}, a graph-based ranking model, where the graph vertices are sentences. We use cosine distance with GloVE embeddings \cite{Glove} as a measure of similarity. The baseline can be thought of as a process of recommendation where one sentence recommends another sentence to which it is similar. The baseline was evaluated on the CNN/DM test set. The oracle is essentially a method that can generate the ground truth summaries and would have a ROUGE score of 1. The results in table \ref{results} show a marginal improvement for actor-critic policy gradient additional self-attention mask model compared to TextRank baseline and Fine-Tuned UniLM. The marginal improvement can be attributed to the bias of the reinforced actor-critic policy gradient towards better ROUGE (AC - Reward).

\begin{table}[h]
\centering
\caption{ROUGE Metrics for Baseline, Fine-tuned UniLM and Our Method (Finetuning UniLM in sync with AC)}
\begin{tabular}{|c|c|c|c|}
\hline
Metrics (F1 score) & Baseline & UniLM & AC + UniLM \\ \hline
ROUGE-1            & 21.0     & 40.79             &   \textbf{41.89}                    \\ \hline
ROUGE-2            & 6.5    & 19.01             &       \textbf{19.22}                \\ \hline
ROUGE-L            & 19.2    & 38.10             &      \textbf{39.28}                \\ \hline
\end{tabular}
\label{results}
\end{table}

\subsection{Hyperparameter Tuning}
Hyperparameter tuning has been a big part for our experiments to achieve generalizability and convergence. A list of hyperparameters we tweaked and tuned for DR.SAS are:
\begin{enumerate}
    \item Max Sequence length - Limited by our local computation resource our initial experiments were done at a sequence length of 384, We eventually switched to 512. This gave us a 0.1 ROUGE score improvement. We attribute this to more context for summarization.

    \item Batch Size - Default: 64, We had to use a batch size of 16 for all our experiments due to compute constraints causing out-of-memory for any batch size beyond. However, we used gradient accumulation in our final experiments to match the default batch size.
    \item Number of epochs - Default: 30, With the limited compute we could only fine-tune for 6 epochs. We compensated for this with a relatively higher learning rate than default.
    \item Learning Rate - Default: 3e-5, We performed random search for the learning rate and settled on 1.5e-4. This gave us relatively faster convergence with fewer epochs.
\end{enumerate}

\subsection{Results Analysis}
 The qualitative analysis show that both UniLM and UniLM + AC produce abstractive summarization of the article by extracting the main points of the article, rewording and connecting the point in coherent way. The UniLM + AC method's inclination to increase the ROUGE (reward) tends to be to add extractive features in the form of details to the output summary. The n-gram overlap helps increase ROUGE. The bias towards a higher ROUGE score may increase the likelihood that the proposed method extracts less essential points depending on the application. \par

Figure \ref{article} presents a snippet of a CNN article about the premier of season five of the HBO show Game of Thrones. The example illustrates how the model pays attention to specific words of the input in generating a multi-sentence summary. The A2C agents uses the attention map for one of the words (state) to select (sample) the attention mask from the learned policy distribution as shown in figure \ref{vizattn}.  \par

 A closer look at the result summaries (figure \ref{quality} of UniLM and UniLM + A2C models show that the RL agents seems to generate masks that would increase emphasis on different parts of the input in the service of increasing the ROUGE score. That add some extractive power to the summary. The model chose to add focus on what 'Doug Gross' has to say about the fifth season premier of game of thrones. Another example in figure \ref{quality2} shows that UniLM + AC increases the ROUGE reward and may extract a bit more factual points but maybe not necessarily essential points. However, the desired level of extracted factual points may be domain and application specific. \par

\begin{figure*}[htp]
\centering
\includegraphics[width=15cm,height=6cm]{article.png}
\caption{Sample Article / Document}
\label{article}
\end{figure*}

\begin{figure*}[htp]
\centering
\includegraphics[width=15cm,height=10cm]{atten_viz.png}
\caption{Visualization of Attention Score Distribution with AC Learning
}
\label{vizattn}
\end{figure*}

\begin{figure*}[htp]
\centering
\includegraphics[width=15cm,height=5cm]{analysis.png}
\caption{Qualitative comparison between UniLM and DR.SAS - Game Of Thrones
}
\label{quality}
\end{figure*}

\begin{figure*}[htp]
\centering
\includegraphics[width=15cm,height=14cm]{art2akka.png}
\caption{Qualitative comparison between UniLM and DR.SAS - Golden State Warriors
}
\label{quality2}
\end{figure*}

\section{Conclusion and Future Work}
We present DR.SAS - a novel method that combines reinforcement learning with a summarization seq-to-seq model to learn dynamic self-attention masks. Our method allows us to generate coherent and factual summaries and address some limitations presented in literature where abstractive summarization models tend to be superfluous in terms of details. Our model achieves better ROUGE scores and also has more factual details in the summaries compared to baseline UniLM model. These learnings can be extended to other natural language tasks. As future work, we would like to experiment with layer specific self-attention to better understand the granularity of features most effective for summarization.
Challenges:
\begin{enumerate}
\item Encoding embeddings or the meaning of tokens in state information would be a more precise way to take the methodology forward
\item A precise state space could lead to a very high number of parameters.
\item UniLM + AC is compute heavy and requires a lot of GPU resources to converge and generalize
The reinforced model utilizes ROUGE metric as a reward. ROUGE is biased towards more overlap between the reference summary and the model generated summary. That may add bias towards extracting more information from the original article (not necessarily essential points)
\end{enumerate}

\section{Contributions and Acknowledgements}
Both authors contributed equally to problem research and formulation, design and coding of the model architecture, development environment setup, data preparation and analysis, designing and conducting experiments, hyper-parameter tuning, evaluation and visual/error analyses and report writing. We are thankful to the CS221 Team for all the support and help throughout and to the folks at Google for the Cloud credits that made this work possible.

redpajama_set_name: RedPajamaArXiv