The UK housing market is looking at a turbulent year ahead. Numerous complications have arisen due to political and economic turmoil, and these have directly impacted how the British public view the housing ladder. Moreover, a few social influences have crept in too, once again determining if and how people move into new property.
Consequently, here are some of the challenges the UK housing market will be dealing with in 2019.
Brexit is most certainly affecting how people are viewing the property market. Brexit means uncertainty nationwide, and most people don't enjoy the sense of gambling or risk when moving to a new house. They need safety, security and assurance; three things Brexit hasn't guaranteed for many people.
When people aren't confident, they don't hand over their money; it's really that simple. Living costs are already high, and most predict they'll get even higher still. Therefore, the last thing people need to be doing amidst all this uncertainty is investing large portions of their hard-earned cash into property. Instead, they're more likely to be found saving and stockpiling.
Other reports indicate a far bleaker scenario for the housing market. The BBC published an article suggesting that the housing market outlook is the worst it's been in two decades, with lack of supply and affordability being the main issues addressed. Also, people are more comfortable extending their current properties if they're strapped for space, rather than buy a completely new property.
The average UK housing price has been steadily rising since 2009. Today, the average property stands at a cost of £230,630, which is not within many people's budgets; especially in the case of first-time buyers and starter homes. When no one is buying, supply also starts to wane, leaving a bottleneck scenario. Both buyers and sellers are stagnant in a stalemate situation, and neither party is willing to budge.
In these kinds of situations, companies like Andrews help buyers to find the property that's right for them and their budget, in a quick and efficient fashion.
There are rays of hope in certain areas of Britain. For example, London reportedly has a 25% proportion of active buyers, which all things considered, isn't too bad at all. Of course, the mild successes of one region isn't necessarily replicated nationwide. London is a very popular city, and if there's any money or opportunity to be found in Britain, it's generally in the capital. Moreover, London is often home to the headquarters of many businesses, meaning more job opportunities and thus, more money to go around. The same isn't true for everywhere.
None of this dispels all the crippling factors affecting property value across the nation; the number of elections and referendums that cause seismic shifts in the economy, taxation targets, business closures and job losses and of course, Brexit. These influences are far reaching in the UK property market, and will remain so for some time.


text:
Divine Nature
From "My Glance Is Clear Like a Sunflower"
by Fernando Pessoa
I believe in the World as in a daisy
Because I see it. But I don't think about it
Because thinking is not understanding...
The World was not made for us to think about
(To think is to be eye-sick)
But for us to look at and be in tune with...
I have no philosophy: I have senses...
If I speak of Nature, it's not because I know what Nature is,
But because I love it, and that's why I love it,
For a lover never knows what he loves,
Why he loves or what love is...
Loving is eternal innocence,
And the only innocence is not to think...
(translated by Alberto Caeiro)
Posted by Anthony F. Chiffolo at 6:54 PM
Anthony F. Chiffolo
From the glaciers of Alaska to the coral reefs of St. Thomas, Anthony F. Chiffolo has been photographing nature in its myriad forms for more than twenty years. His photos of flowers, wildlife, and landscapes have appeared on the covers and in the pages of books such as "Encyclopedia of Animal Behavior," "Shrines of the Holy Land," "Marian Shrines of the United States," "Sadhana," and "Affirmations from the Heart of God." His photography has also been published in pamphlets, brochures, catalogs, and magazines such as "Bugle," "Turkey Call," and "Liguorian." He has exhibited and sold his photos at the prestigious Waterfowl Festival in Easton Maryland; the Fire Island Pines Art Project Biennial Art Show in the Pines, Fire Island; the Hudson River Wildlife Festival in Kingston, New York; and the Chesapeake Bay Foundation's "In Celebration of the Bay" in Annapolis, Maryland. Anthony is also a published author. His books include "Cooking with the Bible: Biblical Foods, Feasts, and Lore" and, forthcoming, "Cooking with the Movies." Please visit his websites and blogs www.anthonychiffolo.com, www.cookingwiththebible.com, and www.cookingwiththemovies.blogspot.com .
Some quotes from Wayne Dyer
Desert Wisdom
Please visit these other blogs or websites
Anthony's website
Cooking with the Bible
Cooking with the Movies
Stephen Thomson, Spiritual Teacher
Pencil Shavings--The Artwork of Terry Miller
The Artwork of Dave Sellers
Design2Share
My Church Lady
SAFE: Spirit and Faith for Everyone
Midwest Green by Wendy Schweigert

redpajama_set_name: RedPajamaCommonCrawl


text:
Hello There, Guest! Login or Register
Staff Statistics Documentaries Game Section Adv. Search	Index	The Help Desk Ban List
The Fringe | Conspiracy, News, Politics, and Fun Forum! › Community Area › General Discussion
« Previous 1 2 3 4 5 ... 838 Next »
Pages (38): « Previous 1 ... 34 35 36 37 38 Next »
BREAKING NEWS * Julian Assange Arrested * UPDATE: US Slaps Assange With 17 More Charges
User Id: 16530
Likes Received: 9,920
RE: BREAKING NEWS * Julian Assange Arrested! * "Merged" (Posted: 04-14-2019, 03:01 PM)
(04-14-2019, 02:34 PM)KOMMA Wrote:
(04-14-2019, 04:20 AM)sivil Wrote: I don't know about the US charges of hacking, not sure how they would prove that, but basically they say he broke in to a computer with Chelsea Manning, and that she would not have been able to do it herself. Penalty is 5 years. Now this Sweden thing was dropped 2 years ago, and they want to extradite him for that. Basically any countries that he leaked embarassing but true information about are now mad at him and some of these "leaders" want revenge. Sorry about those Paradise Papers.....
From what I've heard on-line, they're saying that after Assange was given the first download, Chelsea said she could access more and they're saying that Assange told her how to do it. So what they're saying is that Assange has the ability to hack American state computers. I think in his young life he was a bit of a hacker in Australia, although, my view is that he moved far beyond that low tech that existed in those days. He probably also moved beyond it for credibility reasons and decided to become a publisher for other whistleblowers and hackers. Assange was fully aware of the consequences of hacking and any sensible adult man would've done what Assange did in his position, as he clearly saw there was a market for hacked information that others were willing to provide. WIKILEAKS was born, as a non-profit to bring truth to the world, at the same time as protecting it's sources, and allowing the truth to be known. Assange has a brilliant mind, intelligent enough not to put himself into a position that is above the law.
As such, he broke no laws. He was the editor that looked at material that was provided, like any other editor would do that is working on any investigation. He released that information and the world's media published it. A very clever man, covered his back, by getting all of the journalists that published it involved.
We also have to remember that those newspaper corporations all made money on the back of the information that he released, he was the middle man in this case of investigative journalism.
His Q.C. brought up an important point, that as far as America is concerned, American journalists are covered by the constitution, but foreign journalists and publishers are not. That means that The Guardian newspaper are not covered by the American constitution either, as far as America is concerned. I think you will also find that America was one of the countries that didn't sign up for UN's international Human Rights law. Although someone might like to clarify America's position on that.
https://en.wikipedia.org/wiki/Internatio...rights_law
For instance in the UN International Human Rights Law, there is also a children's charter as they have human rights too. One aspect of that charter is that children have to be asked whether they agree to anything that could impact upon them and their lives. So for instance, if it was explained to a child that using a mobile could then cause cancer to that child, and the child refused that mobile with that knowledge, than it would be illegal for that state to impose that reality onto that child, the same with the parents. Now that is particularly relevant as parents have been imprisoned for malnourishing their children with veganism.
Hence, that is why I wrote about the fact that David Cameron's government was in contravention of the Human Right's law with the 'Bedroom Tax", because the government didn't ask the children if they could remove their bedrooms, when Cameron decided that children would have to share a bedroom. Basically, the state imposed their tax laws, that impacted upon children without their consent. No child would willingly give up their own bedroom, because a politician says that they must. I don't think you will find that the politicians told their own children that they would have to share bedrooms, to move into a smaller property to make way for immigrants that require their bedrooms.
The thing is that most politicians are not aware of human rights law that includes the Children's Charter.
I also remember years ago, Assange saying that the western governments had buried the UN Privacy laws because they are also in contravention of those too.
Liberate the power of love within and it is so powerful, it is a key to life. It requires those that have already liberated their love to help others to do so. Being a liberator of love is wonderful when people appreciate the healing power of it. The authority of the divine key. https://lotusfeet22.blogspot.com
Heir likes this!
Like Post
Security footage shows Julian Assange in his living quarters.
SoldiersAngel likes this!
Labour MP shows support for Julian Assange outside Belmarsh Prison.
Heir, Sami likes this!
Former American BREITBART journalist in England at the Belmarsh Prison where Assange is being held.
Lee Stranahan full speech on Julian Assange's detention 14.14.2019
Sami likes this!
FlyoverCountry
Likes Received: 31,130
http://theindicter.com/my-friend-julian-...in-london/
My Friend Julian Assange – Alicia Castro, ex ambassador in London
I saw my neighbor talking to her cat like it understood her. When I got home I told my dog. We both had a good laugh
KOMMA, Sami, SoldiersAngel likes this!
SkyCat
~ Pontificator Subduer ~
RE: BREAKING NEWS * Julian Assange Arrested! * "Merged" (Posted: 04-15-2019, 10:29 AM)
Quote: On this edition of the program we discuss one and one issue only: the fate of Julian Assange and the fate of journalism itself. CrossTalking with Xavier Moreau, John Laughland, and Alexander Mercouris.
Where there is imbalance I am the counterweight. Beware, for if you are a cause of imbalance you may not enjoy my presence.
likes this!
Quote: The timing was hardly "coincidental." Not long after a trip by Vice President Pence to "convince" Ecuador to hand over Julian Assange, a $4.2 billion IMF loan appears on the scene. Then Ecuador follows through and delivers the Wikileaks publisher to face the wrath of the US political establishment. Do what Washington says, get a bribe. Don't do what Washington says, get a bomb.
Wingsprint
Bill Binney States that the NSA Has 32 Pages of Communications Between Seth Rich and Julian Assange, As Revealed by a FOIA Request
https://governmentslaves.news/2019/04/20...a-request/
About six months ago, a blogpost by "Publius Tacitus" appeared regarding attorney Ty Clevenger's FOIA request regarding Seth Rich:
"But now there is new information that may corroborate what the human sources quoted in the Fox article claimed about Seth's role in getting the DNC documents to Wikileaks. Borne from a FOIA request filed in November 2017 by attorney Ty Clevenger, who requested any information regarding Seth Rich and Julian Assange. The NSA informed Clevenger in a letter dated 4 October 2018 that:
Your request has been processed under the provisions of the FOIA. Fifteen documents (32 pages) responsive to your request have been reviewed by this Agency as required by the FOIA and have found to be currently and properly classified in accordance with Executive Order 13526. These documents meet the criteria for classification as set forth in Subparagraph © of Section 1.4 and remains classified TOP SECRET and SECRET.
If NSA had come back and said, "No, we do not have anything pertaining to Seth Rich," that would have been news. It would have been especially unwelcome news for those who believe that Seth was the source on the DNC emails. But now the opposite is true. The NSA says that it has documents that are classified TS and S. What do those documents say or prove? That remains to be seen."
At the time this appeared, I felt that it was of high significance, but I wasn't quite sure what it meant. What is meant by "any information regarding Julian Assange and Seth Rich?" Reports generated within the NSA that mention both? Communications in which either mentions the other? Direct communications between the two? What was the actual language of Clevenger's request?
The final sentence of the blogpost seems to muddy the waters even more:
Eighth, the NSA has confirmed that it has Top Secret and Secret documents responsive to a FOIA request for information concerning contact between Seth Rich and other people including Julian Assange.
"And other people"?
Although I was somewhat confused by the meaning of this revelation, I commented on its significance. At the very least, it meant that the view that Seth was the source of the Wikileaks DNC releases was more than the brainless and callous conspiracy theory that mainstream media were making it out to be.
oldcynic, SkyCat likes this!
oldcynic
I'll Be a Wise Man Soon
(04-20-2019, 08:22 PM)Wingsprint Wrote: Bill Binney States that the NSA Has 32 Pages of Communications Between Seth Rich and Julian Assange, As Revealed by a FOIA Request
Great find @Wingsprint !!! This cannot be swept under the carpet. Hey John Podesta!? BOOM!
"My Mind, a Field of Battles, Struggles for Peace in a Tight Place."
SkyCat, SoldiersAngel, Wingsprint likes this!
The-Fringe.com is a diverse social, political, conspiracy, and fun web based community where you can participate in active discussions about a variety of topics. Feel free to join in or start a conversation about anything and everything that interests you. Registration is not required but is encouraged in order to fully enjoy all the benefits of being a site member....
© 2015-2018 | Theming By ChillBro Powered By Mybb
Terms Of Service/Fair Use


Seward, Zachary M.. "An extremely expensive cover story — with a new way of footing the bill." Nieman Journalism Lab. Nieman Foundation for Journalism at Harvard, 28 Aug. 2009. Web. 18 Apr. 2019.
Seward, Zachary M.. "An extremely expensive cover story — with a new way of footing the bill." Nieman Journalism Lab. Last modified August 28, 2009. Accessed April 18, 2019. https://www.niemanlab.org/2009/08/an-extremely-expensive-cover-story-with-a-new-way-of-footing-the-bill/.


What child, eager to delay "lights out;" doesn't suddenly become a captive listener at bedtime?
The careful parent who has reserved a little precious energy, can spin golden moments from lights out to slumber. That last bit of time at the close of the day can be cultivated to leaving little minds dancing with lovely memories of the day behind and confident anticipation of the one ahead.
"Tell me all the things I've done today" has become our children's favorite 8:00 P.M. byword. With each child, we review the events of the day, looking for opportunities to praise kindnesses and to recognize well-made choices, to share disappointments and to rethink decisions. Bedtime is a sweet time to lovingly suggest alternatives to misbehavior and to secure pledges for improvement. With daddy holding her little hand and whispering in the glow of a nightlight, our three-year-old resolves with seriousness and dedication to try harder to keep her pants dry.
Even the youngest children delight in their own bedtime routines. Our two-year-old happily closes her eyes when she feels sure we've successfully exhausted the list of all the people who love her. We hope the long list reminds her of the security of her spot among family members at home and gives her a notion of the affection felt for her by loved ones far away. When we complete the list with Heavenly Father and Jesus, she knows that her sphere of association is caring and broad: here at home, 700 miles from here, and even in heaven.
Bedtime can also be a calm, unhurried time to lie beside a thoughtful child and ponder the glorious promises of the gospel. Often our six-year-old lets her mind wander and pricks a willing ear with questions like: "What will I do when I live in Heaven?" "What does it look like there?" "Tell me who I'll see." After patient answers, she closes her eyes, her head filled with promises of eternity. (Her perplexed parents head for the scriptures.) Older children can also learn to cherish these moments when they are encouraged to read at bedtime. Substitute a bedside light and a good book for what might have been thirty minutes of torture for a young insomniac and thirty minutes of frustration for a spent parent.


Best Buy customers often prefer the following products when searching for Coffee Mugs For Cars.
Browse the top-ranked list of Coffee Mugs For Cars below along with associated reviews and opinions.
"It does not leak...Easy to use lock...GREAT FOR CAR...I bought this to keep my my coffee or tea or water warm or cold during the day it works okay nothing special just a cup that keeps things warm or cool for a little bit while longer"
"My wife loved it...Make a good gift...Fits well in car holder and keeps warm....I recommend this bottle for anyone working at a desktop for a long time or in my case, for hot tea"
"Fits well in car holder and keeps warm."
"Make a good gift...My wife loved it...Fits well in car holder and keeps warm....I recommend this bottle for anyone working at a desktop for a long time or in my case, for hot tea"
"You will love ut...The color is nice...Fits perfectly in her car's cup holder....It's not a bad Cup I thought it would keep my coffee hot for a while but it's an average cup but it's nice and small just the right amount of coffee to carry around and it doesn't spill"
"Fits perfectly in her car's cup holder."
"My Wife Loves It...Easy to Clean too...Very stylish and fits comfortably in car cupholder...Always struggle with keeping coffee hot for longs periods of time, this mug works great and it doesn't give you that weird after taste other mugs do when the coffee stays there for a few hours"
"Very stylish and fits comfortably in car cupholder"
"Easy to clean too...gets the job done...Easy to grip fits on car cup holder...I own several of these myself and love them and then when I saw this nice set, I bought it as a birthday gift for a person who doesn't "need" anything...but will certainly get lots of good use from these"
"Easy to grip fits on car cup holder"
"gets the job done...Easy to clean too...Perfect Car Cup...I own several of these myself and love them and then when I saw this nice set, I bought it as a birthday gift for a person who doesn't "need" anything...but will certainly get lots of good use from these"
"Great for the car...I like this design...I love this coffee mug I leave for work at 430 every morning and I pour my coffee at 420 each morning and this mug keeps it hot all day I get off work at 5 in the evening and on my drive home my coffee is still hot"
"Great for the car...I like this design...Even when in a hit car....I love this coffee mug I leave for work at 430 every morning and I pour my coffee at 420 each morning and this mug keeps it hot all day I get off work at 5 in the evening and on my drive home my coffee is still hot"
"Even when in a hit car."
"I LOVE THESE CUPS...No leakage at all...I love that it won't spill in the car while I'm driving and that it keeps everything warm for the whole day!...This mug is a good mug and does its intended job....keeping my coffee warm for a few hours"
"I love that it won't spill in the car while I'm driving and that it keeps everything warm for the whole day!"
"I LOVE THESE CUPS...Nice grip to hold...It fits perfectly in my car's cup holder and your hand can get a good grip on it....I purchased this mug after I saw a commercial for the type of mug and that it vacuum seals with no chance of spilling"
"It fits perfectly in my car's cup holder and your hand can get a good grip on it."
"I will buy again...Great cup for tea...Fits perfect in all the cars....After having a Yeti for years where the air hole would get blocked causing a splash of hot coffee in my face this cup is a nice change, never gets blocked, and being able to choose the chug side versus the sip side is a bonus"
"Fits perfect in all the cars."
"A must have item...Love my Keurig mug...It is cool to the touch and fits in my cars cup holder nicely....We bought this after buying or K-Cup coffee maker and it fits perfectly into the K-Cup coffee maker for a very easy cup of coffee"
"It is cool to the touch and fits in my cars cup holder nicely."
Related to "Coffee Mugs For Cars"


text:
An Open Letter to Ethiopian Listeners on World Press Freedom Day
from the Director of Voice of America
The Voice of America is deeply concerned about the actions taken in late February to jam the shortwave broadcasts of VOA news in Afaan Oromoo, Amharic, and Tigrigna and to block access to VOA web sites in these languages. We have stated publicly that we are opposed to all efforts to interfere with the free flow of news and information.
While we hope to work with the Ethiopian Government to resolve the issues behind the jamming, our broadcasters in Washington and in Ethiopia continue to work diligently to bring you balanced reports, timely news, and major events in Amharic, Afaan Oromoo, and Tigrigna. And, we will bring you full coverage of Ethiopia's national elections.
We are addressing our audience in new ways that did not exist when we began our shortwave radio broadcasts to Ethiopia 29 years ago. We have begun sending e-mails to thousands of people who have written to us in recent months, inviting them to subscribe to our electronic newsletter. Ask someone you know who has access to our web site to enter your e-mail to receive news about Ethiopia Monday through Friday.
You can also now hear our daily radio programs on your TV set. I invite you to tune in to our new and very clear audio transmissions at VOA 24 on Arabsat in Ethiopia during our regular shortwave broadcast hours in all three languages. We have added a morning radio program show in Amharic at 6 a.m. and we have increased the number of shortwave frequencies on which you can receive our morning show. The evening shows are Afaan Oromoo at 8:30 p.m., Amharic at 9 p.m., and Tigrigna at 10 p.m.
Write to us at horn@voanews.com or make a collect call to 202 205 4447. When you hear our VOA musical greeting, press 13 for Afaan Oromoo, 14 for Amharic, or 15 for Tigrigna.
The Voice of America values its audience in Ethiopia. On behalf of our Horn of Africa reporters in the region and all of our writers, editors and technicians in Washington, D.C., I assure you we will do all we can to bring you news you can trust, news you have relied on for almost 30 years.
Danforth W. Austin
Director, Voice of America


Welcome to Caddon View in Innerleithen.
Our Licensed Country Guest House & Restaurant.
Mixing luxury accommodation with wonderful food in a romantic location, a warm welcome awaits you from Lisa and Steve at Caddon View. Enjoy a relaxing break in our beautifuly decorated 4-star licensed country house with restaurant (Tuesday to Saturday, booking essential), in Innerleithen, near Peebles, in the Scottish Borders.
Give yourselves one of the best short breaks in the Tweed Valley - whether it's a long weekend walking the Southern Upland Way; mountain biking at Glentress mountain biking centre; fishing the upper Tweed; golfing at Cardrona or visiting Traquair House, the Scottish Borders is a little bit of undiscovered Scotland.
As quoted in the Sunday Herald: "...The mood inside is relaxing, from the moment Steve greets me at the door to the point when I sit down for dinner, freshly prepared by his wife Lisa. The food would hold its own in a city restaurant, and the menu reflects the makeover the couple have given this Victorian home since taking over in April 2007: fresh, comfortably modern and entirely welcoming."
Originally the Doctor's house, built in the 1850's, Caddon View is now an extremely comfortable and individual bed and breakfast, which feels more like a small hotel, where our personal service and attention to detail in relaxed surroundings are guaranteed. You'll feel a sense of well-being the moment you step inside.


Doheny Blues Festival 2017 is coming to Dana Point!
Posted by Teresa Mihelic HelpDesk on Thursday, May 11th, 2017 at 9:08pm.
Celebrating 20 years this 2017, the Doheny Blues Festival is coming up this May 20 & 21, 2017 and headlined by legendary Eagles guitarist Joe Walsh, Melissa Etheridge's This Is MEmphis Rock & Soul, Jason Bonham's Led Zeppelin Experience, Chris Isaak, 2-time GRAMMY-winner Mavis Staples and Bobby Rush, among many others.
For the last 20 years, the Doheny Blues Festival has created a festival environment like none other! Three alternating stages of blues, rock and soul music, legendary artists, a beachfront venue, microbrews, overnight camping, an eclectic vendor village and 20,000 music fans throughout the weekend! The nationally-recognized festival has welcomed a mix of music legends and rock headliners, and the 2017 lineup will not disappoint!
The layout of the festival offers something for everyone, as two main stages are placed at opposite ends of a large grass field, both within 100 yards of the Pacific Ocean. An intimate, third stage, known as The Backporch, is a great venue to get up-close to your favorite artists. There is also an International Food Court, Microbrew tastings and large Vendor Village. The grass field at the north end of the park (where the festival is held) includes 170 picnic tables, volleyball courts and ample parking.


Two prominent actresses spoke out this week against Disney storylines this week. Keira Knightley, the "Pirates of the Caribbean" star, said she's banned two movies in her house. She said she believes The Little Mermaid and Cinderella are a terrible example to girls. She said Cinderalla is banned "Because she waits around for a rich guy to rescue her." Kristen Bell, who voiced Anna in "Frozen," said she doesn't like how Snow White is kissed. "Don't you think that it's weird that the prince kisses Snow White without her permission?" She said she tells her daughters. "Because you can not kiss someone if they're sleeping!"


text:
Chris M. Kwok, Esq. Joins JAMS in New York Experienced mediator joins New York Resolution Center as a mediator
NEW PANELIST ANNOUNCEMENT
Share Press Release
NEW YORK – JAMS, the largest private provider of mediation and arbitration services worldwide, today announced the addition of Chris M. Kwok, Esq. to its panel. Mr. Kwok will be based in the JAMS New York Resolution Center and will serve as a mediator in Business/Commercial and Employment matters.
Mr. Kwok joins JAMS after 15 years at the U.S. Equal Employment Opportunity Commission (EEOC) in New York, where he mediated hundreds of employment disputes. During his tenure at the EEOC, he also spent four years as supervisory ADR coordinator and oversaw employment law mediations across all EEOC offices within the New York District Office region. In addition to his mediation work at the EEOC, Mr. Kwok spent two years as a federal investigator in the EEOC's Enforcement unit investigating employment discrimination charges.
He was the founding co-chair of both the Labor & Employment Committee at the Asian American Bar Association of New York and the Dispute Resolution Committee at the National Asian Pacific American Bar Association.
Chris has a terrific background and creates solutions to seemingly intractable employment disputes. He adds value to the ADR process by building the bridges necessary to achieve resolution. We're thrilled to have him join our panel.
-Chris Poole, JAMS President and CEO
I have experienced the power and benefit of using ADR to resolve disputes. I make sure each party feels they have been heard and listened to and I look forward to bringing the same level of energy and enthusiasm to my practice at JAMS.
Chirs M. Kowk,Esq., JAMS Mediator
Mr. Kwok received his B.A. from Cornell University and his J.D. from UCLA School of Law.
Founded in 1979, JAMS is the largest private provider of mediation and arbitration services worldwide. With Resolution Centers nationwide and abroad, JAMS and its nearly 350 neutrals are responsible for resolving thousands of the world's important cases. JAMS may be reached at 800-352-5267.
EXPLORE MORE ON THESE TOPICS
RELATED RESOLUTION CENTERS
FEATURED PEOPLE
Chris M. Kwok
Kristine Snyder
JAMS Senior Public Relations & Content Manager
Tel: +1 949-224-4614 (Direct Dial)
Email: Ksnyder@jamsadr.com
Additional Media Contacts ›
Press Kit & Additional Resources ›


text:
Saudi Aramco appoints ex-HSBC CEO Stuart Gulliver to board
Gulliver replaces Sir Mark Moody-Stuart, who had been on the board since 2007
Saudi Aramco appointed former HSBC Holdings CEO Stuart Gulliver to the oil firm's board as part of a reshuffle.
HSBC has had close ties to the kingdom for decades. The bank was the only international adviser to hold a senior role on Aramco's 2019 initial public offering, the world's largest.
In addition to having one of the largest investment banking teams in the kingdom through its Saudi unit, HSBC also owns 31 per cent of Saudi British Bank, the fifth largest lender in the country.
Gulliver replaces Sir Mark Moody-Stuart, who had been on the board since 2007.
Joining Gulliver on the board will be former Aramco CFO Khalid al-Dabbagh, who helped lead the company through the IPO.
Al-Dabbagh, who stepped down as CFO last week, replaces Nabeel Al Amudi, former Transport Minister and currently CEO of Olayan Financing Company.
The appointments bring two people with finance backgrounds onto the board of the world's biggest oil producer as it embarks on a strategy of looking for new ways to raise money from its assets.
Aramco raised $12.4bn earlier this year from the sale of a stake in a subsidiary that holds leasing rights over its oil pipelines, and is also planning to do a similar sale related to its gas pipelines, people familiar told Bloomberg.
The company is also conducting a review of its upstream assets that could result in it bringing in foreign investors, Bloomberg reported in April.
Aramco current board members:
Yasir O. Al-Rumayyan – Chairman
Ibrahim A. Al-Assaf – Vice Chairman
Mohammed A. Al-Jadaan
Mohammad M. Al-Tuwaijri
Khalid H. Al-Dabbagh
Andrew N. Liveris
Lynn Laverty Elsenhans
Peter L. Cella
Mark A. Weinberger
Stuart T. Gulliver
Aramco CEO Amin Nasser also sits on the company's board
aramco energy HSBC Saudi Arabia Stuart Gulliver
Bloomberg July 5, 2021
Logistics, Technology
Starlinks to launch SAR100m hybrid logistics facility in Riyadh
Saudi's LEAP23 announces series of new investments, equalling $580m
Technology, Virtual Reality
Alliance for Metaverse and Web3 Empowerment launches at LEAP23
India energy firms see growth prospect in Africa and Middle East


text:
import sys

from pyspark import since, keyword_only
from pyspark.ml.param.shared import *
from pyspark.ml.util import *
from pyspark.ml.wrapper import JavaEstimator, JavaModel, JavaWrapper
from pyspark.ml.common import inherit_doc
from pyspark.sql import DataFrame

__all__ = ['AFTSurvivalRegression', 'AFTSurvivalRegressionModel',
           'DecisionTreeRegressor', 'DecisionTreeRegressionModel',
           'GBTRegressor', 'GBTRegressionModel',
           'GeneralizedLinearRegression', 'GeneralizedLinearRegressionModel',
           'GeneralizedLinearRegressionSummary', 'GeneralizedLinearRegressionTrainingSummary',
           'IsotonicRegression', 'IsotonicRegressionModel',
           'LinearRegression', 'LinearRegressionModel',
           'LinearRegressionSummary', 'LinearRegressionTrainingSummary',
           'RandomForestRegressor', 'RandomForestRegressionModel']

@inherit_doc
class LinearRegression(JavaEstimator, HasFeaturesCol, HasLabelCol, HasPredictionCol, HasMaxIter,
                       HasRegParam, HasTol, HasElasticNetParam, HasFitIntercept,
                       HasStandardization, HasSolver, HasWeightCol, HasAggregationDepth, HasLoss,
                       JavaMLWritable, JavaMLReadable):
    """
    Linear regression.

    The learning objective is to minimize the specified loss function, with regularization.
    This supports two kinds of loss:

    * squaredError (a.k.a squared loss)
    * huber (a hybrid of squared error for relatively small errors and absolute error for \
    relatively large ones, and we estimate the scale parameter from training data)

    This supports multiple types of regularization:

    * none (a.k.a. ordinary least squares)
    * L2 (ridge regression)
    * L1 (Lasso)
    * L2 + L1 (elastic net)

    Note: Fitting with huber loss only supports none and L2 regularization.

    >>> from pyspark.ml.linalg import Vectors
    >>> df = spark.createDataFrame([
    ...     (1.0, 2.0, Vectors.dense(1.0)),
    ...     (0.0, 2.0, Vectors.sparse(1, [], []))], ["label", "weight", "features"])
    >>> lr = LinearRegression(maxIter=5, regParam=0.0, solver="normal", weightCol="weight")
    >>> model = lr.fit(df)
    >>> test0 = spark.createDataFrame([(Vectors.dense(-1.0),)], ["features"])
    >>> abs(model.transform(test0).head().prediction - (-1.0)) < 0.001
    True
    >>> abs(model.coefficients[0] - 1.0) < 0.001
    True
    >>> abs(model.intercept - 0.0) < 0.001
    True
    >>> test1 = spark.createDataFrame([(Vectors.sparse(1, [0], [1.0]),)], ["features"])
    >>> abs(model.transform(test1).head().prediction - 1.0) < 0.001
    True
    >>> lr.setParams("vector")
    Traceback (most recent call last):
        ...
    TypeError: Method setParams forces keyword arguments.
    >>> lr_path = temp_path + "/lr"
    >>> lr.save(lr_path)
    >>> lr2 = LinearRegression.load(lr_path)
    >>> lr2.getMaxIter()
    5
    >>> model_path = temp_path + "/lr_model"
    >>> model.save(model_path)
    >>> model2 = LinearRegressionModel.load(model_path)
    >>> model.coefficients[0] == model2.coefficients[0]
    True
    >>> model.intercept == model2.intercept
    True
    >>> model.numFeatures
    1
    >>> model.write().format("pmml").save(model_path + "_2")

    .. versionadded:: 1.4.0
    """

    solver = Param(Params._dummy(), "solver", "The solver algorithm for optimization. Supported " +
                   "options: auto, normal, l-bfgs.", typeConverter=TypeConverters.toString)

    loss = Param(Params._dummy(), "loss", "The loss function to be optimized. Supported " +
                 "options: squaredError, huber.", typeConverter=TypeConverters.toString)

    epsilon = Param(Params._dummy(), "epsilon", "The shape parameter to control the amount of " +
                    "robustness. Must be > 1.0. Only valid when loss is huber",
                    typeConverter=TypeConverters.toFloat)

    @keyword_only
    def __init__(self, featuresCol="features", labelCol="label", predictionCol="prediction",
                 maxIter=100, regParam=0.0, elasticNetParam=0.0, tol=1e-6, fitIntercept=True,
                 standardization=True, solver="auto", weightCol=None, aggregationDepth=2,
                 loss="squaredError", epsilon=1.35):
        """
        __init__(self, featuresCol="features", labelCol="label", predictionCol="prediction", \
                 maxIter=100, regParam=0.0, elasticNetParam=0.0, tol=1e-6, fitIntercept=True, \
                 standardization=True, solver="auto", weightCol=None, aggregationDepth=2, \
                 loss="squaredError", epsilon=1.35)
        """
        super(LinearRegression, self).__init__()
        self._java_obj = self._new_java_obj(
            "org.apache.spark.ml.regression.LinearRegression", self.uid)
        self._setDefault(maxIter=100, regParam=0.0, tol=1e-6, loss="squaredError", epsilon=1.35)
        kwargs = self._input_kwargs
        self.setParams(**kwargs)

    @keyword_only
    @since("1.4.0")
    def setParams(self, featuresCol="features", labelCol="label", predictionCol="prediction",
                  maxIter=100, regParam=0.0, elasticNetParam=0.0, tol=1e-6, fitIntercept=True,
                  standardization=True, solver="auto", weightCol=None, aggregationDepth=2,
                  loss="squaredError", epsilon=1.35):
        """
        setParams(self, featuresCol="features", labelCol="label", predictionCol="prediction", \
                  maxIter=100, regParam=0.0, elasticNetParam=0.0, tol=1e-6, fitIntercept=True, \
                  standardization=True, solver="auto", weightCol=None, aggregationDepth=2, \
                  loss="squaredError", epsilon=1.35)
        Sets params for linear regression.
        """
        kwargs = self._input_kwargs
        return self._set(**kwargs)

    def _create_model(self, java_model):
        return LinearRegressionModel(java_model)

    @since("2.3.0")
    def setEpsilon(self, value):
        """
        Sets the value of :py:attr:`epsilon`.
        """
        return self._set(epsilon=value)

    @since("2.3.0")
    def getEpsilon(self):
        """
        Gets the value of epsilon or its default value.
        """
        return self.getOrDefault(self.epsilon)

class LinearRegressionModel(JavaModel, JavaPredictionModel, GeneralJavaMLWritable, JavaMLReadable):
    """
    Model fitted by :class:`LinearRegression`.

    .. versionadded:: 1.4.0
    """

    @property
    @since("2.0.0")
    def coefficients(self):
        """
        Model coefficients.
        """
        return self._call_java("coefficients")

    @property
    @since("1.4.0")
    def intercept(self):
        """
        Model intercept.
        """
        return self._call_java("intercept")

    @property
    @since("2.3.0")
    def scale(self):
        r"""
        The value by which :math:`\|y - X'w\|` is scaled down when loss is "huber", otherwise 1.0.
        """
        return self._call_java("scale")

    @property
    @since("2.0.0")
    def summary(self):
        """
        Gets summary (e.g. residuals, mse, r-squared ) of model on
        training set. An exception is thrown if
        `trainingSummary is None`.
        """
        if self.hasSummary:
            java_lrt_summary = self._call_java("summary")
            return LinearRegressionTrainingSummary(java_lrt_summary)
        else:
            raise RuntimeError("No training summary available for this %s" %
                               self.__class__.__name__)

    @property
    @since("2.0.0")
    def hasSummary(self):
        """
        Indicates whether a training summary exists for this model
        instance.
        """
        return self._call_java("hasSummary")

    @since("2.0.0")
    def evaluate(self, dataset):
        """
        Evaluates the model on a test dataset.

        :param dataset:
          Test dataset to evaluate model on, where dataset is an
          instance of :py:class:`pyspark.sql.DataFrame`
        """
        if not isinstance(dataset, DataFrame):
            raise ValueError("dataset must be a DataFrame but got %s." % type(dataset))
        java_lr_summary = self._call_java("evaluate", dataset)
        return LinearRegressionSummary(java_lr_summary)

class LinearRegressionSummary(JavaWrapper):
    """
    .. note:: Experimental

    Linear regression results evaluated on a dataset.

    .. versionadded:: 2.0.0
    """

    @property
    @since("2.0.0")
    def predictions(self):
        """
        Dataframe outputted by the model's `transform` method.
        """
        return self._call_java("predictions")

    @property
    @since("2.0.0")
    def predictionCol(self):
        """
        Field in "predictions" which gives the predicted value of
        the label at each instance.
        """
        return self._call_java("predictionCol")

    @property
    @since("2.0.0")
    def labelCol(self):
        """
        Field in "predictions" which gives the true label of each
        instance.
        """
        return self._call_java("labelCol")

    @property
    @since("2.0.0")
    def featuresCol(self):
        """
        Field in "predictions" which gives the features of each instance
        as a vector.
        """
        return self._call_java("featuresCol")

    @property
    @since("2.0.0")
    def explainedVariance(self):
        r"""
        Returns the explained variance regression score.
        explainedVariance = :math:`1 - \frac{variance(y - \hat{y})}{variance(y)}`

        .. seealso:: `Wikipedia explain variation
            <http://en.wikipedia.org/wiki/Explained_variation>`_

        .. note:: This ignores instance weights (setting all to 1.0) from
            `LinearRegression.weightCol`. This will change in later Spark
            versions.
        """
        return self._call_java("explainedVariance")

    @property
    @since("2.0.0")
    def meanAbsoluteError(self):
        """
        Returns the mean absolute error, which is a risk function
        corresponding to the expected value of the absolute error
        loss or l1-norm loss.

        .. note:: This ignores instance weights (setting all to 1.0) from
            `LinearRegression.weightCol`. This will change in later Spark
            versions.
        """
        return self._call_java("meanAbsoluteError")

    @property
    @since("2.0.0")
    def meanSquaredError(self):
        """
        Returns the mean squared error, which is a risk function
        corresponding to the expected value of the squared error
        loss or quadratic loss.

        .. note:: This ignores instance weights (setting all to 1.0) from
            `LinearRegression.weightCol`. This will change in later Spark
            versions.
        """
        return self._call_java("meanSquaredError")

    @property
    @since("2.0.0")
    def rootMeanSquaredError(self):
        """
        Returns the root mean squared error, which is defined as the
        square root of the mean squared error.

        .. note:: This ignores instance weights (setting all to 1.0) from
            `LinearRegression.weightCol`. This will change in later Spark
            versions.
        """
        return self._call_java("rootMeanSquaredError")

    @property
    @since("2.0.0")
    def r2(self):
        """
        Returns R^2, the coefficient of determination.

        .. seealso:: `Wikipedia coefficient of determination
            <http://en.wikipedia.org/wiki/Coefficient_of_determination>`_

        .. note:: This ignores instance weights (setting all to 1.0) from
            `LinearRegression.weightCol`. This will change in later Spark
            versions.
        """
        return self._call_java("r2")

    @property
    @since("2.4.0")
    def r2adj(self):
        """
        Returns Adjusted R^2, the adjusted coefficient of determination.

        .. seealso:: `Wikipedia coefficient of determination, Adjusted R^2
            <https://en.wikipedia.org/wiki/Coefficient_of_determination#Adjusted_R2>`_

        .. note:: This ignores instance weights (setting all to 1.0) from
            `LinearRegression.weightCol`. This will change in later Spark versions.
        """
        return self._call_java("r2adj")

    @property
    @since("2.0.0")
    def residuals(self):
        """
        Residuals (label - predicted value)
        """
        return self._call_java("residuals")

    @property
    @since("2.0.0")
    def numInstances(self):
        """
        Number of instances in DataFrame predictions
        """
        return self._call_java("numInstances")

    @property
    @since("2.2.0")
    def degreesOfFreedom(self):
        """
        Degrees of freedom.
        """
        return self._call_java("degreesOfFreedom")

    @property
    @since("2.0.0")
    def devianceResiduals(self):
        """
        The weighted residuals, the usual residuals rescaled by the
        square root of the instance weights.
        """
        return self._call_java("devianceResiduals")

    @property
    @since("2.0.0")
    def coefficientStandardErrors(self):
        """
        Standard error of estimated coefficients and intercept.
        This value is only available when using the "normal" solver.

        If :py:attr:`LinearRegression.fitIntercept` is set to True,
        then the last element returned corresponds to the intercept.

        .. seealso:: :py:attr:`LinearRegression.solver`
        """
        return self._call_java("coefficientStandardErrors")

    @property
    @since("2.0.0")
    def tValues(self):
        """
        T-statistic of estimated coefficients and intercept.
        This value is only available when using the "normal" solver.

        If :py:attr:`LinearRegression.fitIntercept` is set to True,
        then the last element returned corresponds to the intercept.

        .. seealso:: :py:attr:`LinearRegression.solver`
        """
        return self._call_java("tValues")

    @property
    @since("2.0.0")
    def pValues(self):
        """
        Two-sided p-value of estimated coefficients and intercept.
        This value is only available when using the "normal" solver.

        If :py:attr:`LinearRegression.fitIntercept` is set to True,
        then the last element returned corresponds to the intercept.

        .. seealso:: :py:attr:`LinearRegression.solver`
        """
        return self._call_java("pValues")

@inherit_doc
class LinearRegressionTrainingSummary(LinearRegressionSummary):
    """
    .. note:: Experimental

    Linear regression training results. Currently, the training summary ignores the
    training weights except for the objective trace.

    .. versionadded:: 2.0.0
    """

    @property
    @since("2.0.0")
    def objectiveHistory(self):
        """
        Objective function (scaled loss + regularization) at each
        iteration.
        This value is only available when using the "l-bfgs" solver.

        .. seealso:: :py:attr:`LinearRegression.solver`
        """
        return self._call_java("objectiveHistory")

    @property
    @since("2.0.0")
    def totalIterations(self):
        """
        Number of training iterations until termination.
        This value is only available when using the "l-bfgs" solver.

        .. seealso:: :py:attr:`LinearRegression.solver`
        """
        return self._call_java("totalIterations")

@inherit_doc
class IsotonicRegression(JavaEstimator, HasFeaturesCol, HasLabelCol, HasPredictionCol,
                         HasWeightCol, JavaMLWritable, JavaMLReadable):
    """
    Currently implemented using parallelized pool adjacent violators algorithm.
    Only univariate (single feature) algorithm supported.

    >>> from pyspark.ml.linalg import Vectors
    >>> df = spark.createDataFrame([
    ...     (1.0, Vectors.dense(1.0)),
    ...     (0.0, Vectors.sparse(1, [], []))], ["label", "features"])
    >>> ir = IsotonicRegression()
    >>> model = ir.fit(df)
    >>> test0 = spark.createDataFrame([(Vectors.dense(-1.0),)], ["features"])
    >>> model.transform(test0).head().prediction
    0.0
    >>> model.boundaries
    DenseVector([0.0, 1.0])
    >>> ir_path = temp_path + "/ir"
    >>> ir.save(ir_path)
    >>> ir2 = IsotonicRegression.load(ir_path)
    >>> ir2.getIsotonic()
    True
    >>> model_path = temp_path + "/ir_model"
    >>> model.save(model_path)
    >>> model2 = IsotonicRegressionModel.load(model_path)
    >>> model.boundaries == model2.boundaries
    True
    >>> model.predictions == model2.predictions
    True

    .. versionadded:: 1.6.0
    """

    isotonic = \
        Param(Params._dummy(), "isotonic",
              "whether the output sequence should be isotonic/increasing (true) or" +
              "antitonic/decreasing (false).", typeConverter=TypeConverters.toBoolean)
    featureIndex = \
        Param(Params._dummy(), "featureIndex",
              "The index of the feature if featuresCol is a vector column, no effect otherwise.",
              typeConverter=TypeConverters.toInt)

    @keyword_only
    def __init__(self, featuresCol="features", labelCol="label", predictionCol="prediction",
                 weightCol=None, isotonic=True, featureIndex=0):
        """
        __init__(self, featuresCol="features", labelCol="label", predictionCol="prediction", \
                 weightCol=None, isotonic=True, featureIndex=0):
        """
        super(IsotonicRegression, self).__init__()
        self._java_obj = self._new_java_obj(
            "org.apache.spark.ml.regression.IsotonicRegression", self.uid)
        self._setDefault(isotonic=True, featureIndex=0)
        kwargs = self._input_kwargs
        self.setParams(**kwargs)

    @keyword_only
    def setParams(self, featuresCol="features", labelCol="label", predictionCol="prediction",
                  weightCol=None, isotonic=True, featureIndex=0):
        """
        setParams(self, featuresCol="features", labelCol="label", predictionCol="prediction", \
                 weightCol=None, isotonic=True, featureIndex=0):
        Set the params for IsotonicRegression.
        """
        kwargs = self._input_kwargs
        return self._set(**kwargs)

    def _create_model(self, java_model):
        return IsotonicRegressionModel(java_model)

    def setIsotonic(self, value):
        """
        Sets the value of :py:attr:`isotonic`.
        """
        return self._set(isotonic=value)

    def getIsotonic(self):
        """
        Gets the value of isotonic or its default value.
        """
        return self.getOrDefault(self.isotonic)

    def setFeatureIndex(self, value):
        """
        Sets the value of :py:attr:`featureIndex`.
        """
        return self._set(featureIndex=value)

    def getFeatureIndex(self):
        """
        Gets the value of featureIndex or its default value.
        """
        return self.getOrDefault(self.featureIndex)

class IsotonicRegressionModel(JavaModel, JavaMLWritable, JavaMLReadable):
    """
    Model fitted by :class:`IsotonicRegression`.

    .. versionadded:: 1.6.0
    """

    @property
    @since("1.6.0")
    def boundaries(self):
        """
        Boundaries in increasing order for which predictions are known.
        """
        return self._call_java("boundaries")

    @property
    @since("1.6.0")
    def predictions(self):
        """
        Predictions associated with the boundaries at the same index, monotone because of isotonic
        regression.
        """
        return self._call_java("predictions")

class TreeEnsembleParams(DecisionTreeParams):
    """
    Mixin for Decision Tree-based ensemble algorithms parameters.
    """

    subsamplingRate = Param(Params._dummy(), "subsamplingRate", "Fraction of the training data " +
                            "used for learning each decision tree, in range (0, 1].",
                            typeConverter=TypeConverters.toFloat)

    supportedFeatureSubsetStrategies = ["auto", "all", "onethird", "sqrt", "log2"]

    featureSubsetStrategy = \
        Param(Params._dummy(), "featureSubsetStrategy",
              "The number of features to consider for splits at each tree node. Supported " +
              "options: 'auto' (choose automatically for task: If numTrees == 1, set to " +
              "'all'. If numTrees > 1 (forest), set to 'sqrt' for classification and to " +
              "'onethird' for regression), 'all' (use all features), 'onethird' (use " +
              "1/3 of the features), 'sqrt' (use sqrt(number of features)), 'log2' (use " +
              "log2(number of features)), 'n' (when n is in the range (0, 1.0], use " +
              "n * number of features. When n is in the range (1, number of features), use" +
              " n features). default = 'auto'", typeConverter=TypeConverters.toString)

    def __init__(self):
        super(TreeEnsembleParams, self).__init__()

    @since("1.4.0")
    def setSubsamplingRate(self, value):
        """
        Sets the value of :py:attr:`subsamplingRate`.
        """
        return self._set(subsamplingRate=value)

    @since("1.4.0")
    def getSubsamplingRate(self):
        """
        Gets the value of subsamplingRate or its default value.
        """
        return self.getOrDefault(self.subsamplingRate)

    @since("1.4.0")
    def setFeatureSubsetStrategy(self, value):
        """
        Sets the value of :py:attr:`featureSubsetStrategy`.

        .. note:: Deprecated in 2.4.0 and will be removed in 3.0.0.
        """
        return self._set(featureSubsetStrategy=value)

    @since("1.4.0")
    def getFeatureSubsetStrategy(self):
        """
        Gets the value of featureSubsetStrategy or its default value.
        """
        return self.getOrDefault(self.featureSubsetStrategy)

class HasVarianceImpurity(Params):
    """
    Private class to track supported impurity measures.
    """

    supportedImpurities = ["variance"]

    impurity = Param(Params._dummy(), "impurity",
                     "Criterion used for information gain calculation (case-insensitive). " +
                     "Supported options: " +
                     ", ".join(supportedImpurities), typeConverter=TypeConverters.toString)

    def __init__(self):
        super(HasVarianceImpurity, self).__init__()

    @since("1.4.0")
    def setImpurity(self, value):
        """
        Sets the value of :py:attr:`impurity`.
        """
        return self._set(impurity=value)

    @since("1.4.0")
    def getImpurity(self):
        """
        Gets the value of impurity or its default value.
        """
        return self.getOrDefault(self.impurity)

class TreeRegressorParams(HasVarianceImpurity):
    pass

class RandomForestParams(TreeEnsembleParams):
    """
    Private class to track supported random forest parameters.
    """

    numTrees = Param(Params._dummy(), "numTrees", "Number of trees to train (>= 1).",
                     typeConverter=TypeConverters.toInt)

    def __init__(self):
        super(RandomForestParams, self).__init__()

    @since("1.4.0")
    def setNumTrees(self, value):
        """
        Sets the value of :py:attr:`numTrees`.
        """
        return self._set(numTrees=value)

    @since("1.4.0")
    def getNumTrees(self):
        """
        Gets the value of numTrees or its default value.
        """
        return self.getOrDefault(self.numTrees)

class GBTParams(TreeEnsembleParams, HasMaxIter, HasStepSize, HasValidationIndicatorCol):
    """
    Private class to track supported GBT params.
    """

    stepSize = Param(Params._dummy(), "stepSize",
                     "Step size (a.k.a. learning rate) in interval (0, 1] for shrinking " +
                     "the contribution of each estimator.",
                     typeConverter=TypeConverters.toFloat)

    validationTol = Param(Params._dummy(), "validationTol",
                          "Threshold for stopping early when fit with validation is used. " +
                          "If the error rate on the validation input changes by less than the " +
                          "validationTol, then learning will stop early (before `maxIter`). " +
                          "This parameter is ignored when fit without validation is used.",
                          typeConverter=TypeConverters.toFloat)

    @since("3.0.0")
    def getValidationTol(self):
        """
        Gets the value of validationTol or its default value.
        """
        return self.getOrDefault(self.validationTol)

class GBTRegressorParams(GBTParams, TreeRegressorParams):
    """
    Private class to track supported GBTRegressor params.

    .. versionadded:: 3.0.0
    """

    supportedLossTypes = ["squared", "absolute"]

    lossType = Param(Params._dummy(), "lossType",
                     "Loss function which GBT tries to minimize (case-insensitive). " +
                     "Supported options: " + ", ".join(supportedLossTypes),
                     typeConverter=TypeConverters.toString)

    @since("1.4.0")
    def getLossType(self):
        """
        Gets the value of lossType or its default value.
        """
        return self.getOrDefault(self.lossType)

@inherit_doc
class DecisionTreeRegressor(JavaEstimator, HasFeaturesCol, HasLabelCol, HasPredictionCol,
                            DecisionTreeParams, TreeRegressorParams, HasCheckpointInterval,
                            HasSeed, JavaMLWritable, JavaMLReadable, HasVarianceCol):
    """
    `Decision tree <http://en.wikipedia.org/wiki/Decision_tree_learning>`_
    learning algorithm for regression.
    It supports both continuous and categorical features.

    >>> from pyspark.ml.linalg import Vectors
    >>> df = spark.createDataFrame([
    ...     (1.0, Vectors.dense(1.0)),
    ...     (0.0, Vectors.sparse(1, [], []))], ["label", "features"])
    >>> dt = DecisionTreeRegressor(maxDepth=2, varianceCol="variance")
    >>> model = dt.fit(df)
    >>> model.depth
    1
    >>> model.numNodes
    3
    >>> model.featureImportances
    SparseVector(1, {0: 1.0})
    >>> model.numFeatures
    1
    >>> test0 = spark.createDataFrame([(Vectors.dense(-1.0),)], ["features"])
    >>> model.transform(test0).head().prediction
    0.0
    >>> test1 = spark.createDataFrame([(Vectors.sparse(1, [0], [1.0]),)], ["features"])
    >>> model.transform(test1).head().prediction
    1.0
    >>> dtr_path = temp_path + "/dtr"
    >>> dt.save(dtr_path)
    >>> dt2 = DecisionTreeRegressor.load(dtr_path)
    >>> dt2.getMaxDepth()
    2
    >>> model_path = temp_path + "/dtr_model"
    >>> model.save(model_path)
    >>> model2 = DecisionTreeRegressionModel.load(model_path)
    >>> model.numNodes == model2.numNodes
    True
    >>> model.depth == model2.depth
    True
    >>> model.transform(test1).head().variance
    0.0

    .. versionadded:: 1.4.0
    """

    @keyword_only
    def __init__(self, featuresCol="features", labelCol="label", predictionCol="prediction",
                 maxDepth=5, maxBins=32, minInstancesPerNode=1, minInfoGain=0.0,
                 maxMemoryInMB=256, cacheNodeIds=False, checkpointInterval=10, impurity="variance",
                 seed=None, varianceCol=None):
        """
        __init__(self, featuresCol="features", labelCol="label", predictionCol="prediction", \
                 maxDepth=5, maxBins=32, minInstancesPerNode=1, minInfoGain=0.0, \
                 maxMemoryInMB=256, cacheNodeIds=False, checkpointInterval=10, \
                 impurity="variance", seed=None, varianceCol=None)
        """
        super(DecisionTreeRegressor, self).__init__()
        self._java_obj = self._new_java_obj(
            "org.apache.spark.ml.regression.DecisionTreeRegressor", self.uid)
        self._setDefault(maxDepth=5, maxBins=32, minInstancesPerNode=1, minInfoGain=0.0,
                         maxMemoryInMB=256, cacheNodeIds=False, checkpointInterval=10,
                         impurity="variance")
        kwargs = self._input_kwargs
        self.setParams(**kwargs)

    @keyword_only
    @since("1.4.0")
    def setParams(self, featuresCol="features", labelCol="label", predictionCol="prediction",
                  maxDepth=5, maxBins=32, minInstancesPerNode=1, minInfoGain=0.0,
                  maxMemoryInMB=256, cacheNodeIds=False, checkpointInterval=10,
                  impurity="variance", seed=None, varianceCol=None):
        """
        setParams(self, featuresCol="features", labelCol="label", predictionCol="prediction", \
                  maxDepth=5, maxBins=32, minInstancesPerNode=1, minInfoGain=0.0, \
                  maxMemoryInMB=256, cacheNodeIds=False, checkpointInterval=10, \
                  impurity="variance", seed=None, varianceCol=None)
        Sets params for the DecisionTreeRegressor.
        """
        kwargs = self._input_kwargs
        return self._set(**kwargs)

    def _create_model(self, java_model):
        return DecisionTreeRegressionModel(java_model)

@inherit_doc
class DecisionTreeModel(JavaModel, JavaPredictionModel):
    """
    Abstraction for Decision Tree models.

    .. versionadded:: 1.5.0
    """

    @property
    @since("1.5.0")
    def numNodes(self):
        """Return number of nodes of the decision tree."""
        return self._call_java("numNodes")

    @property
    @since("1.5.0")
    def depth(self):
        """Return depth of the decision tree."""
        return self._call_java("depth")

    @property
    @since("2.0.0")
    def toDebugString(self):
        """Full description of model."""
        return self._call_java("toDebugString")

    def __repr__(self):
        return self._call_java("toString")

@inherit_doc
class TreeEnsembleModel(JavaModel):
    """
    (private abstraction)

    Represents a tree ensemble model.
    """

    @property
    @since("2.0.0")
    def trees(self):
        """Trees in this ensemble. Warning: These have null parent Estimators."""
        return [DecisionTreeModel(m) for m in list(self._call_java("trees"))]

    @property
    @since("2.0.0")
    def getNumTrees(self):
        """Number of trees in ensemble."""
        return self._call_java("getNumTrees")

    @property
    @since("1.5.0")
    def treeWeights(self):
        """Return the weights for each tree"""
        return list(self._call_java("javaTreeWeights"))

    @property
    @since("2.0.0")
    def totalNumNodes(self):
        """Total number of nodes, summed over all trees in the ensemble."""
        return self._call_java("totalNumNodes")

    @property
    @since("2.0.0")
    def toDebugString(self):
        """Full description of model."""
        return self._call_java("toDebugString")

    def __repr__(self):
        return self._call_java("toString")

@inherit_doc
class DecisionTreeRegressionModel(DecisionTreeModel, JavaMLWritable, JavaMLReadable):
    """
    Model fitted by :class:`DecisionTreeRegressor`.

    .. versionadded:: 1.4.0
    """

    @property
    @since("2.0.0")
    def featureImportances(self):
        """
        Estimate of the importance of each feature.

        This generalizes the idea of "Gini" importance to other losses,
        following the explanation of Gini importance from "Random Forests" documentation
        by Leo Breiman and Adele Cutler, and following the implementation from scikit-learn.

        This feature importance is calculated as follows:
          - importance(feature j) = sum (over nodes which split on feature j) of the gain,
            where gain is scaled by the number of instances passing through node
          - Normalize importances for tree to sum to 1.

        .. note:: Feature importance for single decision trees can have high variance due to
              correlated predictor variables. Consider using a :py:class:`RandomForestRegressor`
              to determine feature importance instead.
        """
        return self._call_java("featureImportances")

@inherit_doc
class RandomForestRegressor(JavaEstimator, HasFeaturesCol, HasLabelCol, HasPredictionCol, HasSeed,
                            RandomForestParams, TreeRegressorParams, HasCheckpointInterval,
                            JavaMLWritable, JavaMLReadable):
    """
    `Random Forest <http://en.wikipedia.org/wiki/Random_forest>`_
    learning algorithm for regression.
    It supports both continuous and categorical features.

    >>> from numpy import allclose
    >>> from pyspark.ml.linalg import Vectors
    >>> df = spark.createDataFrame([
    ...     (1.0, Vectors.dense(1.0)),
    ...     (0.0, Vectors.sparse(1, [], []))], ["label", "features"])
    >>> rf = RandomForestRegressor(numTrees=2, maxDepth=2, seed=42)
    >>> model = rf.fit(df)
    >>> model.featureImportances
    SparseVector(1, {0: 1.0})
    >>> allclose(model.treeWeights, [1.0, 1.0])
    True
    >>> test0 = spark.createDataFrame([(Vectors.dense(-1.0),)], ["features"])
    >>> model.transform(test0).head().prediction
    0.0
    >>> model.numFeatures
    1
    >>> model.trees
    [DecisionTreeRegressionModel (uid=...) of depth..., DecisionTreeRegressionModel...]
    >>> model.getNumTrees
    2
    >>> test1 = spark.createDataFrame([(Vectors.sparse(1, [0], [1.0]),)], ["features"])
    >>> model.transform(test1).head().prediction
    0.5
    >>> rfr_path = temp_path + "/rfr"
    >>> rf.save(rfr_path)
    >>> rf2 = RandomForestRegressor.load(rfr_path)
    >>> rf2.getNumTrees()
    2
    >>> model_path = temp_path + "/rfr_model"
    >>> model.save(model_path)
    >>> model2 = RandomForestRegressionModel.load(model_path)
    >>> model.featureImportances == model2.featureImportances
    True

    .. versionadded:: 1.4.0
    """

    @keyword_only
    def __init__(self, featuresCol="features", labelCol="label", predictionCol="prediction",
                 maxDepth=5, maxBins=32, minInstancesPerNode=1, minInfoGain=0.0,
                 maxMemoryInMB=256, cacheNodeIds=False, checkpointInterval=10,
                 impurity="variance", subsamplingRate=1.0, seed=None, numTrees=20,
                 featureSubsetStrategy="auto"):
        """
        __init__(self, featuresCol="features", labelCol="label", predictionCol="prediction", \
                 maxDepth=5, maxBins=32, minInstancesPerNode=1, minInfoGain=0.0, \
                 maxMemoryInMB=256, cacheNodeIds=False, checkpointInterval=10, \
                 impurity="variance", subsamplingRate=1.0, seed=None, numTrees=20, \
                 featureSubsetStrategy="auto")
        """
        super(RandomForestRegressor, self).__init__()
        self._java_obj = self._new_java_obj(
            "org.apache.spark.ml.regression.RandomForestRegressor", self.uid)
        self._setDefault(maxDepth=5, maxBins=32, minInstancesPerNode=1, minInfoGain=0.0,
                         maxMemoryInMB=256, cacheNodeIds=False, checkpointInterval=10,
                         impurity="variance", subsamplingRate=1.0, numTrees=20,
                         featureSubsetStrategy="auto")
        kwargs = self._input_kwargs
        self.setParams(**kwargs)

    @keyword_only
    @since("1.4.0")
    def setParams(self, featuresCol="features", labelCol="label", predictionCol="prediction",
                  maxDepth=5, maxBins=32, minInstancesPerNode=1, minInfoGain=0.0,
                  maxMemoryInMB=256, cacheNodeIds=False, checkpointInterval=10,
                  impurity="variance", subsamplingRate=1.0, seed=None, numTrees=20,
                  featureSubsetStrategy="auto"):
        """
        setParams(self, featuresCol="features", labelCol="label", predictionCol="prediction", \
                  maxDepth=5, maxBins=32, minInstancesPerNode=1, minInfoGain=0.0, \
                  maxMemoryInMB=256, cacheNodeIds=False, checkpointInterval=10, \
                  impurity="variance", subsamplingRate=1.0, seed=None, numTrees=20, \
                  featureSubsetStrategy="auto")
        Sets params for linear regression.
        """
        kwargs = self._input_kwargs
        return self._set(**kwargs)

    def _create_model(self, java_model):
        return RandomForestRegressionModel(java_model)

    @since("2.4.0")
    def setFeatureSubsetStrategy(self, value):
        """
        Sets the value of :py:attr:`featureSubsetStrategy`.
        """
        return self._set(featureSubsetStrategy=value)

class RandomForestRegressionModel(TreeEnsembleModel, JavaPredictionModel, JavaMLWritable,
                                  JavaMLReadable):
    """
    Model fitted by :class:`RandomForestRegressor`.

    .. versionadded:: 1.4.0
    """

    @property
    @since("2.0.0")
    def trees(self):
        """Trees in this ensemble. Warning: These have null parent Estimators."""
        return [DecisionTreeRegressionModel(m) for m in list(self._call_java("trees"))]

    @property
    @since("2.0.0")
    def featureImportances(self):
        """
        Estimate of the importance of each feature.

        Each feature's importance is the average of its importance across all trees in the ensemble
        The importance vector is normalized to sum to 1. This method is suggested by Hastie et al.
        (Hastie, Tibshirani, Friedman. "The Elements of Statistical Learning, 2nd Edition." 2001.)
        and follows the implementation from scikit-learn.

        .. seealso:: :py:attr:`DecisionTreeRegressionModel.featureImportances`
        """
        return self._call_java("featureImportances")

@inherit_doc
class GBTRegressor(JavaEstimator, HasFeaturesCol, HasLabelCol, HasPredictionCol,
                   GBTRegressorParams, HasCheckpointInterval, HasSeed, JavaMLWritable,
                   JavaMLReadable):
    """
    `Gradient-Boosted Trees (GBTs) <http://en.wikipedia.org/wiki/Gradient_boosting>`_
    learning algorithm for regression.
    It supports both continuous and categorical features.

    >>> from numpy import allclose
    >>> from pyspark.ml.linalg import Vectors
    >>> df = spark.createDataFrame([
    ...     (1.0, Vectors.dense(1.0)),
    ...     (0.0, Vectors.sparse(1, [], []))], ["label", "features"])
    >>> gbt = GBTRegressor(maxIter=5, maxDepth=2, seed=42)
    >>> print(gbt.getImpurity())
    variance
    >>> print(gbt.getFeatureSubsetStrategy())
    all
    >>> model = gbt.fit(df)
    >>> model.featureImportances
    SparseVector(1, {0: 1.0})
    >>> model.numFeatures
    1
    >>> allclose(model.treeWeights, [1.0, 0.1, 0.1, 0.1, 0.1])
    True
    >>> test0 = spark.createDataFrame([(Vectors.dense(-1.0),)], ["features"])
    >>> model.transform(test0).head().prediction
    0.0
    >>> test1 = spark.createDataFrame([(Vectors.sparse(1, [0], [1.0]),)], ["features"])
    >>> model.transform(test1).head().prediction
    1.0
    >>> gbtr_path = temp_path + "gbtr"
    >>> gbt.save(gbtr_path)
    >>> gbt2 = GBTRegressor.load(gbtr_path)
    >>> gbt2.getMaxDepth()
    2
    >>> model_path = temp_path + "gbtr_model"
    >>> model.save(model_path)
    >>> model2 = GBTRegressionModel.load(model_path)
    >>> model.featureImportances == model2.featureImportances
    True
    >>> model.treeWeights == model2.treeWeights
    True
    >>> model.trees
    [DecisionTreeRegressionModel (uid=...) of depth..., DecisionTreeRegressionModel...]
    >>> validation = spark.createDataFrame([(0.0, Vectors.dense(-1.0))],
    ...              ["label", "features"])
    >>> model.evaluateEachIteration(validation, "squared")
    [0.0, 0.0, 0.0, 0.0, 0.0]
    >>> gbt = gbt.setValidationIndicatorCol("validationIndicator")
    >>> gbt.getValidationIndicatorCol()
    'validationIndicator'
    >>> gbt.getValidationTol()
    0.01

    .. versionadded:: 1.4.0
    """

    @keyword_only
    def __init__(self, featuresCol="features", labelCol="label", predictionCol="prediction",
                 maxDepth=5, maxBins=32, minInstancesPerNode=1, minInfoGain=0.0,
                 maxMemoryInMB=256, cacheNodeIds=False, subsamplingRate=1.0,
                 checkpointInterval=10, lossType="squared", maxIter=20, stepSize=0.1, seed=None,
                 impurity="variance", featureSubsetStrategy="all", validationTol=0.01,
                 validationIndicatorCol=None):
        """
        __init__(self, featuresCol="features", labelCol="label", predictionCol="prediction", \
                 maxDepth=5, maxBins=32, minInstancesPerNode=1, minInfoGain=0.0, \
                 maxMemoryInMB=256, cacheNodeIds=False, subsamplingRate=1.0, \
                 checkpointInterval=10, lossType="squared", maxIter=20, stepSize=0.1, seed=None, \
                 impurity="variance", featureSubsetStrategy="all", validationTol=0.01, \
                 validationIndicatorCol=None)
        """
        super(GBTRegressor, self).__init__()
        self._java_obj = self._new_java_obj("org.apache.spark.ml.regression.GBTRegressor", self.uid)
        self._setDefault(maxDepth=5, maxBins=32, minInstancesPerNode=1, minInfoGain=0.0,
                         maxMemoryInMB=256, cacheNodeIds=False, subsamplingRate=1.0,
                         checkpointInterval=10, lossType="squared", maxIter=20, stepSize=0.1,
                         impurity="variance", featureSubsetStrategy="all", validationTol=0.01)
        kwargs = self._input_kwargs
        self.setParams(**kwargs)

    @keyword_only
    @since("1.4.0")
    def setParams(self, featuresCol="features", labelCol="label", predictionCol="prediction",
                  maxDepth=5, maxBins=32, minInstancesPerNode=1, minInfoGain=0.0,
                  maxMemoryInMB=256, cacheNodeIds=False, subsamplingRate=1.0,
                  checkpointInterval=10, lossType="squared", maxIter=20, stepSize=0.1, seed=None,
                  impuriy="variance", featureSubsetStrategy="all", validationTol=0.01,
                  validationIndicatorCol=None):
        """
        setParams(self, featuresCol="features", labelCol="label", predictionCol="prediction", \
                  maxDepth=5, maxBins=32, minInstancesPerNode=1, minInfoGain=0.0, \
                  maxMemoryInMB=256, cacheNodeIds=False, subsamplingRate=1.0, \
                  checkpointInterval=10, lossType="squared", maxIter=20, stepSize=0.1, seed=None, \
                  impurity="variance", featureSubsetStrategy="all", validationTol=0.01, \
                  validationIndicatorCol=None)
        Sets params for Gradient Boosted Tree Regression.
        """
        kwargs = self._input_kwargs
        return self._set(**kwargs)

    def _create_model(self, java_model):
        return GBTRegressionModel(java_model)

    @since("1.4.0")
    def setLossType(self, value):
        """
        Sets the value of :py:attr:`lossType`.
        """
        return self._set(lossType=value)

    @since("2.4.0")
    def setFeatureSubsetStrategy(self, value):
        """
        Sets the value of :py:attr:`featureSubsetStrategy`.
        """
        return self._set(featureSubsetStrategy=value)

    @since("3.0.0")
    def setValidationIndicatorCol(self, value):
        """
        Sets the value of :py:attr:`validationIndicatorCol`.
        """
        return self._set(validationIndicatorCol=value)

class GBTRegressionModel(TreeEnsembleModel, JavaPredictionModel, JavaMLWritable, JavaMLReadable):
    """
    Model fitted by :class:`GBTRegressor`.

    .. versionadded:: 1.4.0
    """

    @property
    @since("2.0.0")
    def featureImportances(self):
        """
        Estimate of the importance of each feature.

        Each feature's importance is the average of its importance across all trees in the ensemble
        The importance vector is normalized to sum to 1. This method is suggested by Hastie et al.
        (Hastie, Tibshirani, Friedman. "The Elements of Statistical Learning, 2nd Edition." 2001.)
        and follows the implementation from scikit-learn.

        .. seealso:: :py:attr:`DecisionTreeRegressionModel.featureImportances`
        """
        return self._call_java("featureImportances")

    @property
    @since("2.0.0")
    def trees(self):
        """Trees in this ensemble. Warning: These have null parent Estimators."""
        return [DecisionTreeRegressionModel(m) for m in list(self._call_java("trees"))]

    @since("2.4.0")
    def evaluateEachIteration(self, dataset, loss):
        """
        Method to compute error or loss for every iteration of gradient boosting.

        :param dataset:
            Test dataset to evaluate model on, where dataset is an
            instance of :py:class:`pyspark.sql.DataFrame`
        :param loss:
            The loss function used to compute error.
            Supported options: squared, absolute
        """
        return self._call_java("evaluateEachIteration", dataset, loss)

@inherit_doc
class AFTSurvivalRegression(JavaEstimator, HasFeaturesCol, HasLabelCol, HasPredictionCol,
                            HasFitIntercept, HasMaxIter, HasTol, HasAggregationDepth,
                            JavaMLWritable, JavaMLReadable):
    """
    .. note:: Experimental

    Accelerated Failure Time (AFT) Model Survival Regression

    Fit a parametric AFT survival regression model based on the Weibull distribution
    of the survival time.

    .. seealso:: `AFT Model <https://en.wikipedia.org/wiki/Accelerated_failure_time_model>`_

    >>> from pyspark.ml.linalg import Vectors
    >>> df = spark.createDataFrame([
    ...     (1.0, Vectors.dense(1.0), 1.0),
    ...     (1e-40, Vectors.sparse(1, [], []), 0.0)], ["label", "features", "censor"])
    >>> aftsr = AFTSurvivalRegression()
    >>> model = aftsr.fit(df)
    >>> model.predict(Vectors.dense(6.3))
    1.0
    >>> model.predictQuantiles(Vectors.dense(6.3))
    DenseVector([0.0101, 0.0513, 0.1054, 0.2877, 0.6931, 1.3863, 2.3026, 2.9957, 4.6052])
    >>> model.transform(df).show()
    +-------+---------+------+----------+
    |  label| features|censor|prediction|
    +-------+---------+------+----------+
    |    1.0|    [1.0]|   1.0|       1.0|
    |1.0E-40|(1,[],[])|   0.0|       1.0|
    +-------+---------+------+----------+
    ...
    >>> aftsr_path = temp_path + "/aftsr"
    >>> aftsr.save(aftsr_path)
    >>> aftsr2 = AFTSurvivalRegression.load(aftsr_path)
    >>> aftsr2.getMaxIter()
    100
    >>> model_path = temp_path + "/aftsr_model"
    >>> model.save(model_path)
    >>> model2 = AFTSurvivalRegressionModel.load(model_path)
    >>> model.coefficients == model2.coefficients
    True
    >>> model.intercept == model2.intercept
    True
    >>> model.scale == model2.scale
    True

    .. versionadded:: 1.6.0
    """

    censorCol = Param(Params._dummy(), "censorCol",
                      "censor column name. The value of this column could be 0 or 1. " +
                      "If the value is 1, it means the event has occurred i.e. " +
                      "uncensored; otherwise censored.", typeConverter=TypeConverters.toString)
    quantileProbabilities = \
        Param(Params._dummy(), "quantileProbabilities",
              "quantile probabilities array. Values of the quantile probabilities array " +
              "should be in the range (0, 1) and the array should be non-empty.",
              typeConverter=TypeConverters.toListFloat)
    quantilesCol = Param(Params._dummy(), "quantilesCol",
                         "quantiles column name. This column will output quantiles of " +
                         "corresponding quantileProbabilities if it is set.",
                         typeConverter=TypeConverters.toString)

    @keyword_only
    def __init__(self, featuresCol="features", labelCol="label", predictionCol="prediction",
                 fitIntercept=True, maxIter=100, tol=1E-6, censorCol="censor",
                 quantileProbabilities=list([0.01, 0.05, 0.1, 0.25, 0.5, 0.75, 0.9, 0.95, 0.99]),
                 quantilesCol=None, aggregationDepth=2):
        """
        __init__(self, featuresCol="features", labelCol="label", predictionCol="prediction", \
                 fitIntercept=True, maxIter=100, tol=1E-6, censorCol="censor", \
                 quantileProbabilities=[0.01, 0.05, 0.1, 0.25, 0.5, 0.75, 0.9, 0.95, 0.99], \
                 quantilesCol=None, aggregationDepth=2)
        """
        super(AFTSurvivalRegression, self).__init__()
        self._java_obj = self._new_java_obj(
            "org.apache.spark.ml.regression.AFTSurvivalRegression", self.uid)
        self._setDefault(censorCol="censor",
                         quantileProbabilities=[0.01, 0.05, 0.1, 0.25, 0.5, 0.75, 0.9, 0.95, 0.99],
                         maxIter=100, tol=1E-6)
        kwargs = self._input_kwargs
        self.setParams(**kwargs)

    @keyword_only
    @since("1.6.0")
    def setParams(self, featuresCol="features", labelCol="label", predictionCol="prediction",
                  fitIntercept=True, maxIter=100, tol=1E-6, censorCol="censor",
                  quantileProbabilities=list([0.01, 0.05, 0.1, 0.25, 0.5, 0.75, 0.9, 0.95, 0.99]),
                  quantilesCol=None, aggregationDepth=2):
        """
        setParams(self, featuresCol="features", labelCol="label", predictionCol="prediction", \
                  fitIntercept=True, maxIter=100, tol=1E-6, censorCol="censor", \
                  quantileProbabilities=[0.01, 0.05, 0.1, 0.25, 0.5, 0.75, 0.9, 0.95, 0.99], \
                  quantilesCol=None, aggregationDepth=2):
        """
        kwargs = self._input_kwargs
        return self._set(**kwargs)

    def _create_model(self, java_model):
        return AFTSurvivalRegressionModel(java_model)

    @since("1.6.0")
    def setCensorCol(self, value):
        """
        Sets the value of :py:attr:`censorCol`.
        """
        return self._set(censorCol=value)

    @since("1.6.0")
    def getCensorCol(self):
        """
        Gets the value of censorCol or its default value.
        """
        return self.getOrDefault(self.censorCol)

    @since("1.6.0")
    def setQuantileProbabilities(self, value):
        """
        Sets the value of :py:attr:`quantileProbabilities`.
        """
        return self._set(quantileProbabilities=value)

    @since("1.6.0")
    def getQuantileProbabilities(self):
        """
        Gets the value of quantileProbabilities or its default value.
        """
        return self.getOrDefault(self.quantileProbabilities)

    @since("1.6.0")
    def setQuantilesCol(self, value):
        """
        Sets the value of :py:attr:`quantilesCol`.
        """
        return self._set(quantilesCol=value)

    @since("1.6.0")
    def getQuantilesCol(self):
        """
        Gets the value of quantilesCol or its default value.
        """
        return self.getOrDefault(self.quantilesCol)

class AFTSurvivalRegressionModel(JavaModel, JavaMLWritable, JavaMLReadable):
    """
    .. note:: Experimental

    Model fitted by :class:`AFTSurvivalRegression`.

    .. versionadded:: 1.6.0
    """

    @property
    @since("2.0.0")
    def coefficients(self):
        """
        Model coefficients.
        """
        return self._call_java("coefficients")

    @property
    @since("1.6.0")
    def intercept(self):
        """
        Model intercept.
        """
        return self._call_java("intercept")

    @property
    @since("1.6.0")
    def scale(self):
        """
        Model scale parameter.
        """
        return self._call_java("scale")

    @since("2.0.0")
    def predictQuantiles(self, features):
        """
        Predicted Quantiles
        """
        return self._call_java("predictQuantiles", features)

    @since("2.0.0")
    def predict(self, features):
        """
        Predicted value
        """
        return self._call_java("predict", features)

@inherit_doc
class GeneralizedLinearRegression(JavaEstimator, HasLabelCol, HasFeaturesCol, HasPredictionCol,
                                  HasFitIntercept, HasMaxIter, HasTol, HasRegParam, HasWeightCol,
                                  HasSolver, JavaMLWritable, JavaMLReadable):
    """
    .. note:: Experimental

    Generalized Linear Regression.

    Fit a Generalized Linear Model specified by giving a symbolic description of the linear
    predictor (link function) and a description of the error distribution (family). It supports
    "gaussian", "binomial", "poisson", "gamma" and "tweedie" as family. Valid link functions for
    each family is listed below. The first link function of each family is the default one.

    * "gaussian" -> "identity", "log", "inverse"

    * "binomial" -> "logit", "probit", "cloglog"

    * "poisson"  -> "log", "identity", "sqrt"

    * "gamma"    -> "inverse", "identity", "log"

    * "tweedie"  -> power link function specified through "linkPower". \
                    The default link power in the tweedie family is 1 - variancePower.

    .. seealso:: `GLM <https://en.wikipedia.org/wiki/Generalized_linear_model>`_

    >>> from pyspark.ml.linalg import Vectors
    >>> df = spark.createDataFrame([
    ...     (1.0, Vectors.dense(0.0, 0.0)),
    ...     (1.0, Vectors.dense(1.0, 2.0)),
    ...     (2.0, Vectors.dense(0.0, 0.0)),
    ...     (2.0, Vectors.dense(1.0, 1.0)),], ["label", "features"])
    >>> glr = GeneralizedLinearRegression(family="gaussian", link="identity", linkPredictionCol="p")
    >>> model = glr.fit(df)
    >>> transformed = model.transform(df)
    >>> abs(transformed.head().prediction - 1.5) < 0.001
    True
    >>> abs(transformed.head().p - 1.5) < 0.001
    True
    >>> model.coefficients
    DenseVector([1.5..., -1.0...])
    >>> model.numFeatures
    2
    >>> abs(model.intercept - 1.5) < 0.001
    True
    >>> glr_path = temp_path + "/glr"
    >>> glr.save(glr_path)
    >>> glr2 = GeneralizedLinearRegression.load(glr_path)
    >>> glr.getFamily() == glr2.getFamily()
    True
    >>> model_path = temp_path + "/glr_model"
    >>> model.save(model_path)
    >>> model2 = GeneralizedLinearRegressionModel.load(model_path)
    >>> model.intercept == model2.intercept
    True
    >>> model.coefficients[0] == model2.coefficients[0]
    True

    .. versionadded:: 2.0.0
    """

    family = Param(Params._dummy(), "family", "The name of family which is a description of " +
                   "the error distribution to be used in the model. Supported options: " +
                   "gaussian (default), binomial, poisson, gamma and tweedie.",
                   typeConverter=TypeConverters.toString)
    link = Param(Params._dummy(), "link", "The name of link function which provides the " +
                 "relationship between the linear predictor and the mean of the distribution " +
                 "function. Supported options: identity, log, inverse, logit, probit, cloglog " +
                 "and sqrt.", typeConverter=TypeConverters.toString)
    linkPredictionCol = Param(Params._dummy(), "linkPredictionCol", "link prediction (linear " +
                              "predictor) column name", typeConverter=TypeConverters.toString)
    variancePower = Param(Params._dummy(), "variancePower", "The power in the variance function " +
                          "of the Tweedie distribution which characterizes the relationship " +
                          "between the variance and mean of the distribution. Only applicable " +
                          "for the Tweedie family. Supported values: 0 and [1, Inf).",
                          typeConverter=TypeConverters.toFloat)
    linkPower = Param(Params._dummy(), "linkPower", "The index in the power link function. " +
                      "Only applicable to the Tweedie family.",
                      typeConverter=TypeConverters.toFloat)
    solver = Param(Params._dummy(), "solver", "The solver algorithm for optimization. Supported " +
                   "options: irls.", typeConverter=TypeConverters.toString)
    offsetCol = Param(Params._dummy(), "offsetCol", "The offset column name. If this is not set " +
                      "or empty, we treat all instance offsets as 0.0",
                      typeConverter=TypeConverters.toString)

    @keyword_only
    def __init__(self, labelCol="label", featuresCol="features", predictionCol="prediction",
                 family="gaussian", link=None, fitIntercept=True, maxIter=25, tol=1e-6,
                 regParam=0.0, weightCol=None, solver="irls", linkPredictionCol=None,
                 variancePower=0.0, linkPower=None, offsetCol=None):
        """
        __init__(self, labelCol="label", featuresCol="features", predictionCol="prediction", \
                 family="gaussian", link=None, fitIntercept=True, maxIter=25, tol=1e-6, \
                 regParam=0.0, weightCol=None, solver="irls", linkPredictionCol=None, \
                 variancePower=0.0, linkPower=None, offsetCol=None)
        """
        super(GeneralizedLinearRegression, self).__init__()
        self._java_obj = self._new_java_obj(
            "org.apache.spark.ml.regression.GeneralizedLinearRegression", self.uid)
        self._setDefault(family="gaussian", maxIter=25, tol=1e-6, regParam=0.0, solver="irls",
                         variancePower=0.0)
        kwargs = self._input_kwargs

        self.setParams(**kwargs)

    @keyword_only
    @since("2.0.0")
    def setParams(self, labelCol="label", featuresCol="features", predictionCol="prediction",
                  family="gaussian", link=None, fitIntercept=True, maxIter=25, tol=1e-6,
                  regParam=0.0, weightCol=None, solver="irls", linkPredictionCol=None,
                  variancePower=0.0, linkPower=None, offsetCol=None):
        """
        setParams(self, labelCol="label", featuresCol="features", predictionCol="prediction", \
                  family="gaussian", link=None, fitIntercept=True, maxIter=25, tol=1e-6, \
                  regParam=0.0, weightCol=None, solver="irls", linkPredictionCol=None, \
                  variancePower=0.0, linkPower=None, offsetCol=None)
        Sets params for generalized linear regression.
        """
        kwargs = self._input_kwargs
        return self._set(**kwargs)

    def _create_model(self, java_model):
        return GeneralizedLinearRegressionModel(java_model)

    @since("2.0.0")
    def setFamily(self, value):
        """
        Sets the value of :py:attr:`family`.
        """
        return self._set(family=value)

    @since("2.0.0")
    def getFamily(self):
        """
        Gets the value of family or its default value.
        """
        return self.getOrDefault(self.family)

    @since("2.0.0")
    def setLinkPredictionCol(self, value):
        """
        Sets the value of :py:attr:`linkPredictionCol`.
        """
        return self._set(linkPredictionCol=value)

    @since("2.0.0")
    def getLinkPredictionCol(self):
        """
        Gets the value of linkPredictionCol or its default value.
        """
        return self.getOrDefault(self.linkPredictionCol)

    @since("2.0.0")
    def setLink(self, value):
        """
        Sets the value of :py:attr:`link`.
        """
        return self._set(link=value)

    @since("2.0.0")
    def getLink(self):
        """
        Gets the value of link or its default value.
        """
        return self.getOrDefault(self.link)

    @since("2.2.0")
    def setVariancePower(self, value):
        """
        Sets the value of :py:attr:`variancePower`.
        """
        return self._set(variancePower=value)

    @since("2.2.0")
    def getVariancePower(self):
        """
        Gets the value of variancePower or its default value.
        """
        return self.getOrDefault(self.variancePower)

    @since("2.2.0")
    def setLinkPower(self, value):
        """
        Sets the value of :py:attr:`linkPower`.
        """
        return self._set(linkPower=value)

    @since("2.2.0")
    def getLinkPower(self):
        """
        Gets the value of linkPower or its default value.
        """
        return self.getOrDefault(self.linkPower)

    @since("2.3.0")
    def setOffsetCol(self, value):
        """
        Sets the value of :py:attr:`offsetCol`.
        """
        return self._set(offsetCol=value)

    @since("2.3.0")
    def getOffsetCol(self):
        """
        Gets the value of offsetCol or its default value.
        """
        return self.getOrDefault(self.offsetCol)

class GeneralizedLinearRegressionModel(JavaModel, JavaPredictionModel, JavaMLWritable,
                                       JavaMLReadable):
    """
    .. note:: Experimental

    Model fitted by :class:`GeneralizedLinearRegression`.

    .. versionadded:: 2.0.0
    """

    @property
    @since("2.0.0")
    def coefficients(self):
        """
        Model coefficients.
        """
        return self._call_java("coefficients")

    @property
    @since("2.0.0")
    def intercept(self):
        """
        Model intercept.
        """
        return self._call_java("intercept")

    @property
    @since("2.0.0")
    def summary(self):
        """
        Gets summary (e.g. residuals, deviance, pValues) of model on
        training set. An exception is thrown if
        `trainingSummary is None`.
        """
        if self.hasSummary:
            java_glrt_summary = self._call_java("summary")
            return GeneralizedLinearRegressionTrainingSummary(java_glrt_summary)
        else:
            raise RuntimeError("No training summary available for this %s" %
                               self.__class__.__name__)

    @property
    @since("2.0.0")
    def hasSummary(self):
        """
        Indicates whether a training summary exists for this model
        instance.
        """
        return self._call_java("hasSummary")

    @since("2.0.0")
    def evaluate(self, dataset):
        """
        Evaluates the model on a test dataset.

        :param dataset:
          Test dataset to evaluate model on, where dataset is an
          instance of :py:class:`pyspark.sql.DataFrame`
        """
        if not isinstance(dataset, DataFrame):
            raise ValueError("dataset must be a DataFrame but got %s." % type(dataset))
        java_glr_summary = self._call_java("evaluate", dataset)
        return GeneralizedLinearRegressionSummary(java_glr_summary)

class GeneralizedLinearRegressionSummary(JavaWrapper):
    """
    .. note:: Experimental

    Generalized linear regression results evaluated on a dataset.

    .. versionadded:: 2.0.0
    """

    @property
    @since("2.0.0")
    def predictions(self):
        """
        Predictions output by the model's `transform` method.
        """
        return self._call_java("predictions")

    @property
    @since("2.0.0")
    def predictionCol(self):
        """
        Field in :py:attr:`predictions` which gives the predicted value of each instance.
        This is set to a new column name if the original model's `predictionCol` is not set.
        """
        return self._call_java("predictionCol")

    @property
    @since("2.2.0")
    def numInstances(self):
        """
        Number of instances in DataFrame predictions.
        """
        return self._call_java("numInstances")

    @property
    @since("2.0.0")
    def rank(self):
        """
        The numeric rank of the fitted linear model.
        """
        return self._call_java("rank")

    @property
    @since("2.0.0")
    def degreesOfFreedom(self):
        """
        Degrees of freedom.
        """
        return self._call_java("degreesOfFreedom")

    @property
    @since("2.0.0")
    def residualDegreeOfFreedom(self):
        """
        The residual degrees of freedom.
        """
        return self._call_java("residualDegreeOfFreedom")

    @property
    @since("2.0.0")
    def residualDegreeOfFreedomNull(self):
        """
        The residual degrees of freedom for the null model.
        """
        return self._call_java("residualDegreeOfFreedomNull")

    @since("2.0.0")
    def residuals(self, residualsType="deviance"):
        """
        Get the residuals of the fitted model by type.

        :param residualsType: The type of residuals which should be returned.
                              Supported options: deviance (default), pearson, working, and response.
        """
        return self._call_java("residuals", residualsType)

    @property
    @since("2.0.0")
    def nullDeviance(self):
        """
        The deviance for the null model.
        """
        return self._call_java("nullDeviance")

    @property
    @since("2.0.0")
    def deviance(self):
        """
        The deviance for the fitted model.
        """
        return self._call_java("deviance")

    @property
    @since("2.0.0")
    def dispersion(self):
        """
        The dispersion of the fitted model.
        It is taken as 1.0 for the "binomial" and "poisson" families, and otherwise
        estimated by the residual Pearson's Chi-Squared statistic (which is defined as
        sum of the squares of the Pearson residuals) divided by the residual degrees of freedom.
        """
        return self._call_java("dispersion")

    @property
    @since("2.0.0")
    def aic(self):
        """
        Akaike's "An Information Criterion"(AIC) for the fitted model.
        """
        return self._call_java("aic")

@inherit_doc
class GeneralizedLinearRegressionTrainingSummary(GeneralizedLinearRegressionSummary):
    """
    .. note:: Experimental

    Generalized linear regression training results.

    .. versionadded:: 2.0.0
    """

    @property
    @since("2.0.0")
    def numIterations(self):
        """
        Number of training iterations.
        """
        return self._call_java("numIterations")

    @property
    @since("2.0.0")
    def solver(self):
        """
        The numeric solver used for training.
        """
        return self._call_java("solver")

    @property
    @since("2.0.0")
    def coefficientStandardErrors(self):
        """
        Standard error of estimated coefficients and intercept.

        If :py:attr:`GeneralizedLinearRegression.fitIntercept` is set to True,
        then the last element returned corresponds to the intercept.
        """
        return self._call_java("coefficientStandardErrors")

    @property
    @since("2.0.0")
    def tValues(self):
        """
        T-statistic of estimated coefficients and intercept.

        If :py:attr:`GeneralizedLinearRegression.fitIntercept` is set to True,
        then the last element returned corresponds to the intercept.
        """
        return self._call_java("tValues")

    @property
    @since("2.0.0")
    def pValues(self):
        """
        Two-sided p-value of estimated coefficients and intercept.

        If :py:attr:`GeneralizedLinearRegression.fitIntercept` is set to True,
        then the last element returned corresponds to the intercept.
        """
        return self._call_java("pValues")

    def __repr__(self):
        return self._call_java("toString")

if __name__ == "__main__":
    import doctest
    import pyspark.ml.regression
    from pyspark.sql import SparkSession
    globs = pyspark.ml.regression.__dict__.copy()
    # The small batch size here ensures that we see multiple batches,
    # even in these small test examples:
    spark = SparkSession.builder\
        .master("local[2]")\
        .appName("ml.regression tests")\
        .getOrCreate()
    sc = spark.sparkContext
    globs['sc'] = sc
    globs['spark'] = spark
    import tempfile
    temp_path = tempfile.mkdtemp()
    globs['temp_path'] = temp_path
    try:
        (failure_count, test_count) = doctest.testmod(globs=globs, optionflags=doctest.ELLIPSIS)
        spark.stop()
    finally:
        from shutil import rmtree
        try:
            rmtree(temp_path)
        except OSError:
            pass
    if failure_count:
        sys.exit(-1)

redpajama_set_name: RedPajamaGithub