text:
Israel & Zionism: A Contemporary Challenge
A History of Reconstructionist Zionism
Rabbi David Teutsch, Ph.D.
On Occupied Ground
Rabbi Rebecca Lillian
Finding a New Narrative
Rabbi Toba Spitzer
Values, Middle East Politics, and the Future of Israel
Robert Prybutok
Israel Task Force
In preparing this essay, I had the opportunity to read books, editorials and articles from an 80-year span of Reconstructionist history.[1] This wide-ranging array of material reflects a striking uniformity in Reconstructionist positions on Zionism. From the movement's outset, every intellectual leader has been a committed Zionist loyal to the same principles. Their responses to changing political situations have been quite predictable based on their earlier positions. Their frequent discussion of issues regarding Palestine and then Israel indicates their centrality in the history of Reconstructionist Judaism.
There are many legitimate dates that one can use for the beginning of the Reconstructionist movement. Often cited is the founding of the first Reconstructionist synagogue, the Society for the Advancement of Judaism (SAJ) in 1922. For the purpose of this essay, I will begin in 1934-5, which includes the twin events launching Reconstructionist Judaism as a North American movement with a distinctive ideology: the publication of Rabbi Mordecai Kaplan's magnum opus, Judaism as a Civilization, and the launching of The Reconstructionist magazine.
THE PURPOSE OF ZIONISM
Rabbi Kaplan's thought formed the original basis for Reconstructionism, and that is true also of Reconstructionist Zionism. He wrote, "Judaism is unlikely to survive, either as an ancillary or coordinate civilization, unless it thrive as a primary civilization in Palestine."[2] At a time when most of the American Jewish community was indifferent to Jewish efforts to build Eretz Yisrael (the land of Israel), Kaplan imagined a majority Jewish homeland in Palestine where Jewish life would normalize and Jewish culture would thrive, and he saw activities to further those efforts as central to the future of the Jewish people. In the old Reconstructionist seal, Eretz Yisrael in Hebrew was at the center. Before 1948, that was translated as Palestine, and after 1948, it was simply transliterated. "State of Israel" never appeared on the seal. In Kaplan's view, the establishment of a Jewish state was not an end in itself.
The only way in which the return of Jews to Eretz Yisrael can come to mean the fulfillment of the long nurtured hope of the Jewish People is for the Jewish People to prepare itself for that fulfillment by undergoing what is tantamount to a metamorphosis. Without a Jewish People regenerated in sprit, no matter how successful the state...Zion will continue to be unredeemed.[3]
Following some of the thought of Ahad Ha'am, Kaplan understood the Diaspora and Israel to be deeply intertwined, with each requiring the other. He expected the cultural revival in Israel to spread through the Diaspora, and he believed that the moral voice of the Diaspora would save Israel from becoming merely another Levantine state, a state characterized by authoritarian government, a lack of full civil liberties and a tribal, premodern orientation.
The first issue of The Reconstructionist contains an extensive statement of purpose by the full editorial board, including Mordecai Kaplan, that deals with Zionism:
We consider the establishment of Palestine indispensable to the life of Judaism in the diaspora. We seek to enable Jewish civilization so to root itself in the soil of Palestine as to make of that land the cultural center for Israel's intellectual and spiritual rebirth. We oppose any attempt to render Palestine the object of imperialist aims or the victim of private profit-seeking. We endorse every effort to reward the establishment of a cooperative commonwealth in Palestine based upon social justice and social cooperation.[4]
The last words of that statement remain critically important. From its earliest days Reconstructionist Zionism has been committed not just to establishing a Jewish homeland but to developing a Jewish homeland characterized by the fulfillment of the Jewish tradition's vision of social justice, which in the Reconstructionist understanding includes equality for all citizens, peace with its neighbors and care for the needs of the less fortunate.
Unlike many Zionists, Reconstructionists did not see the goal of Zionism as creating a Jewish nation to which everyone would move. Reconstructionists have always seen the Diaspora Jewish communities in general, and the one in the United States in particular, as playing a vital role in Jewish life. In the words of influential educator and editorial board member Samuel Dinin in The Reconstructionist:
We must drive home the realization to Jews the world over that Palestine and the Diaspora are joint partners in a common endeavor. One without the other, one to the exclusion of the other, one at the expense of the other, will lead to the demoralization of both. What we are trying to do is not to establish one kind of life in the Diaspora and another in Palestine. We are trying to establish a new type of Jewish life for Palestine and the Diaspora. The richness and the vitality of this new life will depend upon the extent to which both are interdependent, and sustain each other. Just as much as Palestine is the affair and the creation of the Diaspora, so Palestine must realize that the Diaspora is its affair too.[5]
An editorial several years into the existence of the State of Israel continued to emphasize that the fulfillment of the Zionist dream entails changes for the whole Jewish people:
Zionism is the effort to insure the unity of the Jewish people and the continuity of its civilization under conditions of maximum freedom and creativity. To this end it strives for the security, stability and prosperity of Israel as the cultural center of world Jewry, for the democratic organization of Jewish community life in the Diaspora, and for the maintenance of a continuous interchange of cultural and spiritual influences between Israel and the Diaspora, so that the collective life of the Jewish people can help Jews everywhere to fulfill themselves as human beings and can contribute to their welfare and that of their fellow men.[6]
Mordecai Kaplan was distressed with the failure of Zionism to evolve once the State of Israel came into existence. In 1953 he lectured,
Should Jewish civilization fail to be at home in Eretz Yisrael, it will disappear everywhere else. Should it disappear everywhere else, it is bound to give way to some new Levantine civilization in Eretz Yisrael.[7]
A 1988 resolution of the Federation of Reconstructionist Congregations and Havurot (which later became the Jewish Reconstructionist Federation) made a point of saying that Jews in North America have an obligation to directly address ethical issues that concern Israel because our fates are intertwined. The 2004 Israel Task Force report ratified by the Jewish Reconstructionist Federation (JRF) board made a point of stating the mutual obligations of Diaspora and Israeli Jewry while affirming the positions described throughout this essay.
THE ARAB POPULATION
From the outset of Aliya to Palestine, the question of how to relate to the Arab population that was already there was central. The Reconstructionist commitment was to buying land and settling on it, not taking any land by force. Reconstructionist leaders argued that the Jewish nationalism of the halutzim (pioneers) could provide a model to the downtrodden Arab population dominated by an Arab upper class for developing an Arab nationalism that would help the downtrodden come into their own, and that the two nationalisms could co-exist.
We must make the Arab masses understand that the political ambitions of Arab nationalism can be realized in the vast territories in the east where the bulk of Arabs reside, but that in Palestine, Arab nationalism, like Jewish nationalism, must be so adapted as to make possible the peaceful cooperation of two national groups.[8]
They were actively arguing against the Jewish Revisionists led by Ze'ev Jabotinsky; from the earliest years, the Reconstructionist position has always insisted on full rights for both Jews and Arabs.
We Jews have suffered sufficiently from fascist tyranny. We shall have suffered in vain unless that suffering shall have taught us to appreciate tolerance, democratic procedure, and civil liberty. Incidents ... will only tend to alienate the progressive elements of the world—and those of Jewry included—from the cause of Zionism.[9]
Continued Arab violence stood as a serious threat to the Jewish settlements of Palestine throughout this period. That did not alter the movement's approach, which rests on fundamental moral commitments.
The Reconstructionist has repeatedly maintained that lasting peace in Palestine will not be realized until Zionists formulate a clear and unequivocal program of cooperation with the Arab people. The events of the last months have revealed that such a program is indispensable.[10]
Despite the ongoing and dangerous friction between Arabs and Jews in Palestine, the Reconstructionist position remained that providing equal treatment and full civil liberties would eventually lead to peace. However, equal treatment never occurred. In 1949 the movement objected to consideration of a blue and white flag with a star of David at its center because it saw such a flag as making Arab citizens feel less than equal. "It must be remembered that the Israeli flag is to represent the common national aspiration of all the citizens of Israel."[11] Later, the magazine protested unequal treatment in the form of unbalanced investment in roads, sewers and schools in Arab villages when compared to Jewish ones.
THE VALUES THAT SHOULD SHAPE A JEWISH GOVERNMENT
There was a strong affinity for many years between the Reconstructionist and Labor Zionist movements because of their shared values. Even during difficult times, their approach in the pre-state days was similar. The fusion of Zionism with democratic socialism were seen as keys to the vitality of the State. Zionism and democratic socialism
combined the ideals of nationhood and of social justice. This synthesis unloosed a reservoir of energy and sacrifice, of social and cultural creativity. They had found the key to their own salvation, and had stumbled upon what might become the key to the social salvation of other peoples.[12]
As the danger to European Jews grew during the 1930s, the need for Jewish refugees to enter Palestine became ever more urgent.
We do not bring these considerations before our readers in order to further depress their spirits at a time when faith in the future and courage to bear present ills are so badly needed. We mention them in order to direct the attention of our readers to what we believe to be the essential prerequisites to a solution of the refugee problem, particularly as it affects the Jews. These are 1) the establishment of a world order founded on democracy, social and economic justice, freedom of religion and culture, and international cooperation and 2) the recognition of the right of every nation to a national home adequate to its vital needs and, specifically, the right of the Jewish people to its national home in Palestine. To these ends we must direct our efforts and never permit the spirit of defeatism to slacken them.[13]
In Reconstructionist thought, the urgent need for a Jewish state and the danger to Jewish lives did not justify a suspension of social justice. Quite the reverse: for Reconstructionist ideology the long-term safety of a Jewish state is dependent upon the triumph of liberty and social justice not only for Israelis but for Arabs as well. While achieving that ideal has remained an unmet hope since its first Reconstructionist expression in the 1930s, the movement continued to advocate for it even when Labor Zionist governments failed to live up to its ideals.
This attitude explains the movement's powerful opposition to the Irgun and Jabotinsky before the state was created.
It is time for the true nature of the terrorist movement in Palestine to be exposed. It is not merely an uprising against Great Britain. It is an armed revolt against the democratically elected and responsible Jewish leadership, a quasi-Fascist attempt to seize power by force and to discredit the legitimate leaders of Jewry.[14]
Of course the Reconstructionist movement fully celebrated the coming into existence of the State of Israel in 1948. The magazine quoted extensively from the Declaration of Independence, which promised,
The State of Israel...will be based on freedom, justice and peace as envisaged by the prophets of Israel; it will ensure complete equality of social and political rights to all its inhabitants irrespective of religion, race or sex; it will guarantee freedom of religion, conscience, language, education, and culture.
The magazine's editors made clear their understanding of this celebratory moment:
In the spiritual exaltation with which Jews everywhere greeted the news of the declaration of Israel's independence and of its recognition by the United States, one sensed an awareness of the transcendent importance of the historic occasion. Something more than a mere change in political situation had taken place. Our generation has had the zekhut [merit] to witness and bear testimony to a triumphant act of faith, to the successful assertion of the spiritual forces making for freedom, justice and peace in the face of apparently insuperable obstacles.[15]
Jewish nationalism and the existence of a Jewish state are not ends in themselves. The full meaning of Zionism is to be found in the assertion of values and the revitalization of the Jewish people.
The State of Israel does not coincide with the Jewish People, neither is it co-extensive with the whole of Jewry. We Jews have to maintain our historic position that a state is not the supreme form of human association. Only those who are actively united for the furtherance of universal freedom, justice and peace, whether they be few or many, constitute the supreme form of human association.[16]
Given these values, which include a commitment to religious pluralism, the Reconstructionist movement has always objected to the entanglement of the Israeli government with religious affairs. Exemption from the army for yeshiva students, state funding for Orthodox rabbis, and the institution of the Chief Rabbinate all undermine any effort toward religious pluralism. This church-state entanglement affects the lives of every Jew who lives in Israel, as the Chief Rabbinate controls whether any Jew can marry in the country.
Our own position on the religious issue in Israel has been frequently expressed. We are opposed to all forms of religious authoritarianism and clericalism. We believe that religion must operate thru persuasion and not by the police power of the state.[17]
Kaplan worried about too much entanglement of church and state on other grounds as well:
Now that Israel is to be the homeland of the Jewish People and its civilization, it will have to foster the kind of Jewish religion that can afford to be voluntaristic and that will renounce all ambition to engage in power politics. Without having a voice in the administration of the State of Israel, voluntaristic Jewish religion, which is likely to be diverse in belief and practice, can enhance the Jewish way of life in Israel and in the Diaspora.[18]
These values have continuously shaped Reconstructionist commitments to the State of Israel. A clear summary can be found in a Reconstructionist Rabbinical Association (RRA) plenary resolution passed in 1998:
The Reconstructionist Rabbinical Association hereby rededicates itself and our members to the vision of the State of Israel as outlined in its Declaration of Independence: "The State of Israel...will be based on freedom, justice and peace as envisaged by the prophets of Israel; it will ensure complete equality of social and political rights to all its inhabitants irrespective of religion, race or sex; it will guarantee freedom of religion, conscience, language, education, and culture.
ISRAEL AND THE WORLD
On the eve of statehood, it was clear that the emerging State of Israel could not succeed by itself. It depended then—and still depends—on the support of other countries committed to democracy. The role of the United States in particular is seen as vital for the survival not only of the Jewish state but of democratic values in the Middle East.
If we Americans are really committed to the advancement of democracy in the world, as the only concept of government which can insure peace and prosperity, then we should be equally committed to combating all forms of anti-democratic government....Once again, as in 1933, the Jews are the touchstone of the world's responsibility. The democracies stood by then when the Nazis declared war against the Jews; they were compelled ultimately to fight for their very lives. Today, the Arabs are testing the integrity of the democracies; if they have their way with the Jews, the U.N. will cease to be a power, and mankind will enter upon the road leading straight to the next—and perhaps final—war.[19]
The Reconstructionist movement has long recognized the strong affinity between Israel and the democracies of Europe and North America. As a small nation, Israel is dependent upon its allies for weaponry, intelligence and political support. The movement is committed to supporting these alliances.
The Reconstructionist movement has always recognized the right of the State of Israel to exist within secure and recognized borders and has urged continuous dialogue between Israel and its neighbors toward that end. In taking this position, the movement has repeatedly acknowledged the power and enmity of several Arab nations and of Iran as well. Nevertheless, following the Six Day War and the acquisition of the West Bank and Gaza in 1967, Rabbi Jack Cohen, former senior rabbi of the Society for the Advancement of Judaism, former director of the Jewish Reconstructionist Foundation, and already five years into his nearly 30-year term as Hillel Director at Hebrew University, argued in the Israeli press that the Israeli army should immediately give back the West Bank to Jordan because nothing but strife and enmity could come from retaining it. Since 1967, Reconstructionists have steadily opposed West Bank and Gaza Jewish settlements as long-term impediments to peace.
There is a commonplace of strategic thinking about Israel: of democracy, a Jewish state and holding onto the occupied territories, Israel can have any two, but not all three. The positions of the Reconstructionist movement have at every turn supported a democratic Jewish state, which means finding a way to avoid holding onto the occupied territories. This is understood as serving both Jewish and Arab interests, though how to achieve this end has never been clear. Several statements have supported U.S. efforts at peace making. After the Camp David accords were signed, an editorial in The Reconstructionist commented, "None but the most callous can help rejoice" at the treaty with Egypt in which Israel agreed to withdraw from Sinai in exchange for peace. That editorial particularly lauded President Jimmy Carter's role, along with those of Prime Minister Menahem Begin and President Anwar Sadat.
In 1988 the RRA plenum passed a resolution that is indicative of this direction:
The Reconstructionist Rabbinical Association expresses its support for those elements in the Israeli population and government which believe that direct negotiations with the representatives of the Palestinian people are imperative if the conflict between Israel, the Palestinians, and the Arab states is to be resolved peacefully and justly. We believe that participation in a peace process such as has been suggested by the United States during Secretary of State Shultz's recent trip to the Middle East is less of a threat to Israel than the continued control of the territories and of the over one million Palestinians therein.
We recognize that Israel has the ultimate responsibility to make decisions regarding her security. Yet we express our conviction that Israel's security is ultimately dependent on the achievement of a negotiated resolution rather than on perpetual control of the territories. We further believe that the unique partnership between Israel and the North American Jewish community mandates that we share our concerns with Israel, and that we do so out of love and respect for Israel as the Jewish homeland.
This resolution indicates the clear sense of the movement that our fate and that of Israel are so intertwined that we have an obligation to address each other honestly and directly, with American Jews appealing directly to their allies in Israel to form common cause for the future of the Jewish people. As Kaplan put it, "World Jewry without Eretz Yisrael is like a soul without a body; Eretz Yisrael without World Jewry is like a body without a soul."[20]
In 2000, the JRF responded to the Second Intifada in this way:
At this time, we reaffirm the JRF Resolution on Israel, which states, in part: "We call for a just and lasting peace that will protect Israel's right to a secure existence and that will also fulfill the legitimate national aspirations of the Palestinian people. Such a peace will require Palestinian leaders and heads of Arab governments at long last to acknowledge Israel as a permanent state in the region and to renounce all violence directed against the Jewish homeland." We therefore urge the Palestinian leadership to halt the violence and return to the negotiating table. We also urge Israel to address the legitimate social and economic grievances of its Arab citizens, as well as to pursue negotiations and an end to the cycle of violence with the Palestinian people.
All the organs of the movement express the same themes in dealing with the pursuit of peace. Even given these shared principles and values, however, there can still be sharp strategic differences regarding assessments of the military and political situation at any given moment. Those differences can lead to different conclusions about best strategies for guaranteeing Israel's safety and vitality and for bringing peace between Israel, Palestinians and the Arab nations of the region.
[1]The author is grateful to Rachael Burgess for doing research that bolstered this essay, and particularly grateful to Rabbi Deborah Waxman, who generously shared some of her unpublished doctoral research.
[2] Mordecai M. Kaplan, Judaism as a Civilization: Toward a Reconstruction of American Jewish Life, (Macmillan, 1934; Schocken Books, 1972), p. 273. Page citations from the Schocken edition.
[3] Mordecai M. Kaplan, A New Zionism, (Herzl Press &Reconstructionist Press, 1959), 99. 11-12. The contents of most of the book are lectures delivered in 1954; this second edition has one lecture delivered in 1958.
[4] The Reconstructionist 1.1, January 1935, p. 4.
[5] The Reconstructionist 2.6, May 1, 1936, p. 13.
[6] The Reconstructionist 18.17, December 26, 1952, p. 4.
[7] Mordecai M. Kaplan, A New Zionism (Herzl Press and Jewish Reconstructionist Press, 1959), p. 42.
[8] The Reconstructionist 2.7, May 15, 1936, p. 4.
[9] The Reconstructionist 3.6, April 30, 1937, p. 4.
[10] The Reconstructionist 3.2 March 5, 1937, p. 3.
[11] The Reconstructionist 15.1, February 18, 1949, p. 5.
[12] The Reconstructionist 14.7, May 14, 1948, p. 4.
[13] The Reconstructionist 5.14, November 10, 1939, p. 5.
[14] The Reconstructionist 13.8, May 30, 1947, pp. 5-6.
[16] The Reconstructionist, 15.7, May 13, 1949.
[18] Mordecai M. Kaplan, A New Zionism, p. 92.
[19] The Reconstructionist 14.5, April 16, 1948, p. 4.
[20] Mordecai M. Kaplan, A New Zionism, p. 139.
David Teutsch, Ph.D., is a man in motion, a multi-tasker with many interests. Trained as a rabbi, he earned his doctorate from the Wharton School at the University of Pennsylvania; is past president of RRC; and currently leads RRC's Center for Jewish Ethics.


text:
Q: pairwise distances between list of strings rows within a dataframe column i have a dataframe that has a column of lists of string ids. (see below).
I want to create a distance matrix between all pairwise "distances" between all the rows
(e.g. if 10 rows, then it's a 10x 10 matrix).
the rows are lists of ids, so I'm not sure how things like pdist can be used.
the values are string ids. just like string names

ids
0   [58545-19, 462423-43, 277581-25]
1   [0]
2   [454950-82, 433701-46, 228790-63, 266250-52, 458759-98, 152986-78, 222217-39, 433515-16, 265589-83, 439403-23, 277892-38, 223497-19, 224072-83, 461887-57, 436147-12, 227479-78, 228893-32, 279415-18, 439426-27, 437742-46, 438156-73, 438458-68, 277898-05, 438675-76, 454658-95, 431222-77, 462579-94, 434939-86, 222211-09, 178215-13, 459566-11, 463200-04, 439278-94, 459505-18, 399139-66, 455735-62, 327382-03, 439040-62, 233779-51, 431387-38, 438589-72, 437892-49, 458178-76]
3   [431380-63]
4   [442539-01, 434388-16, 454950-82, 463197-61, 228893-32, 464322-07, 462579-94, 438781-51, 437273-11, 265395-79, 463560-76, 462525-31, 439426-27, 438458-68, 464300-38, 442676-80]
5   [234729-10, 435926-98, 416670-04, 179514-28]
6   [0]
7   [0]
8   [267726-25, 235217-71, 227314-72, 185293-18, 434447-56, 170271-19, 454661-20]
9   [0]

A: Here is a solution using the scipy.spatial.distance.pdist function to compute the pairwise distances (see full code at the end).
Step by step
custom jaccard function
While scipy.spatial.distance has a jaccard method, this one is made for boolean arrays. We will need to define a custom function (using this definition of the jaccard distance: 1-intersection/union):
def jaccard(u, v):
    u,v = set(u[0]), set(v[0]) # pdist will pass 2D data [[a,b,c]], so we need to slice
    return 1-len(u.intersection(v))/len(u.union(v))

Then we apply it on our dataframe column.
Warning: pdist expects a multidimensional array as input (Series won't work), so we need to slice the column as DataFrame (df[['ids']]). Also, passing directly the function as metric would cause an error as the function is not vectorized (see comment on that point below), so we need to wrap it in a lambda.
pdist(df[['ids']], metric=lambda u,v: jaccard(u,v))

As mentioned above, it is also possible to pass a vectorized function instead. For this, we can use numpy.vectorize. Note that the function is slightly different than previously. Here we do not slice the first element of the passed values as it is already 1D.
def jaccard(u, v):
    u,v = set(u), set(v)
    return 1-len(u.intersection(v))/len(u.union(v))

pdist(df[['ids']], metric=np.vectorize(jaccard))

NB. A quick test on the provided dataset showed that the vectorized approach is actually slower than the lambda.
output as 2D
Finally, we transform the output back to matrix using scipy.spatial.distance.squareform and the pandas.DataFrame constructor:
pd.DataFrame(squareform(pdist(df[['ids']], metric=lambda u,v: jaccard(u,v))))

Example (full code)
Let's start from this input:
df = pd.DataFrame([[['58545-19', '462423-43', '277581-25']],
                   [['0']],
                   [['454950-82', '433701-46', '228790-63', '266250-52', '458759-98', '152986-78', '222217-39', '433515-16', '265589-83', '439403-23', '277892-38', '223497-19', '224072-83', '461887-57', '436147-12', '227479-78', '228893-32', '279415-18', '439426-27', '437742-46', '438156-73', '438458-68', '277898-05', '438675-76', '454658-95', '431222-77', '462579-94', '434939-86', '222211-09', '178215-13', '459566-11', '463200-04', '439278-94', '459505-18', '399139-66', '455735-62', '327382-03', '439040-62', '233779-51', '431387-38', '438589-72', '437892-49', '458178-76']],
                   [['431380-63']],
                   [['442539-01', '434388-16', '454950-82', '463197-61', '228893-32', '464322-07', '462579-94', '438781-51', '437273-11', '265395-79', '463560-76', '462525-31', '439426-27', '438458-68', '464300-38', '442676-80']],
                   [['234729-10', '435926-98', '416670-04', '179514-28']],
                   [['0']],
                   [['0']],
                   [['267726-25', '235217-71', '227314-72', '185293-18', '434447-56', '170271-19', '454661-20']],
                   [['0']],
                  ], columns=['ids'])

from scipy.spatial.distance import pdist, squareform

def jaccard(u, v):
    u,v = set(u[0]), set(v[0])
    return 1-len(u.intersection(v))/len(u.union(v))

pd.DataFrame(squareform(pdist(df[['ids']], metric=lambda u,v: jaccard(u,v))))

output:
     0    1         2    3         4    5    6    7    8    9
0  0.0  1.0  1.000000  1.0  1.000000  1.0  1.0  1.0  1.0  1.0
1  1.0  0.0  1.000000  1.0  1.000000  1.0  0.0  0.0  1.0  0.0
2  1.0  1.0  0.000000  1.0  0.907407  1.0  1.0  1.0  1.0  1.0
3  1.0  1.0  1.000000  0.0  1.000000  1.0  1.0  1.0  1.0  1.0
4  1.0  1.0  0.907407  1.0  0.000000  1.0  1.0  1.0  1.0  1.0
5  1.0  1.0  1.000000  1.0  1.000000  0.0  1.0  1.0  1.0  1.0
6  1.0  0.0  1.000000  1.0  1.000000  1.0  0.0  0.0  1.0  0.0
7  1.0  0.0  1.000000  1.0  1.000000  1.0  0.0  0.0  1.0  0.0
8  1.0  1.0  1.000000  1.0  1.000000  1.0  1.0  1.0  0.0  1.0
9  1.0  0.0  1.000000  1.0  1.000000  1.0  0.0  0.0  1.0  0.0

Here is a graphical representation of the distances for the provided dataset (white = further away):

A: If you want to compute the Jaccard distance between the lists, thus based on the number of items in common, you can iterate through the rows, compute the dissimilarity and then construct your distances DataFrame. Moreover, since the resulting DataFrame will be symmetric, to optimize the computation you can construct just the upper triangle and then copy it into the lower triangle to create the full DataFrame.
Starting from a Dataframe df containing the IDs, you can do this in the following way:
def jaccard(a, b):
    a, b = set(a), set(b)
    c = a.intersection(b)
    return 1 - float(len(c)) / (len(a) + len(b) - len(c))

distances = pd.DataFrame(columns=range(df.shape[0]))

for i in range(0, len(df)):
    for j in range(i, len(df)):
        distances.loc[i, j] = jaccard(df['ids'].iloc[i],df['ids'].iloc[j])

distances[distances.isnull()] = distances.transpose()

redpajama_set_name: RedPajamaStackExchange


Earlier this summer, we had the privilege of coordinating Jill and Minh's wedding at Vintage Villas in Austin, TX. Their sweet ceremony was held in the manicured garden below which has a breathtaking view overlooking Lake Travis. Jill and Minh are an adorable couple – they met at work, in a "Scrubs" like story…Jill is a Nurse and Minh is a Surgeon. :) Needless to say, the two of them are beyond busy and needed some help to pull together their big day – so, we created a custom package for them where we provided Timeline Management, Event Design, and Vendor Selection in addition to "Month-Of" services. The couple has a love for water and sailboats (Minh even proposed on a sailboat!) so we used a few nautical elements throughout the wedding. Their day turned out so beautiful! We wish them all the best as they move to Baltimore next year for Minh's Residency. Congrats you two!
They are SO cute. I love the brown touches with the blue.


text:
Home Innovation Smartphones Mobility
LG Pay will be a no show at MWC: Report
LG's yet-to-be launched mobile payment system will take a backseat to allow flagship phone the G5 hog the spotlight at the upcoming Mobile World Congress, a South Korean report says.
Written by Cho Mu-Hyun, Contributing Writer on Feb. 11, 2016
LG Electronics will not show off its mobile payment system LG Pay at the upcoming Mobile World Congress in Barcelona, in order to put the full focus on the G5, a report from South Korea claims.
According to DigitalDaily, the South Korean tech giant has postponed the unveiling of LG Pay to the second quarter of the year to "concentrate" on the G5.
Latest news on Asia
India 'anti-colonial' to its economic detriment: Marc Andreessen in incoherent Twitter rant
Singtel shrinks net profit by SG$16m
Facebook withdraws Free Basics project in India
Samsung to provide public safety network in South Korea
LG Uplus, the mobile carrier affiliate of LG Electronics, was due to go to Barcelona to display the mobile payment system, but its display area had been removed, South Korean media said, citing an unnamed LG official.
A separate report earlier claimed that LG was planning to unveil a universal payment card called LG White Card at the mobile tradeshow.
South Korea's second-largest electronics firm put in fingerprint recognition on last year's flagship V10, speculated to be in preparation for the launch of its own mobile payment app.
In November last year, it signed agreements with local credit card firms Shinhan Card and KB Kookmin Card for mutual cooperation in the area.
Apple and Samsung have already launched payment services in the last year and are heavily promoting them globally.
According to TrendForce, global mobile payment will grow to $620 billion in 2016, a growth of 37.8 percent from 2015's $450 billion.
LG revealed that its new phone will be called the G5 when it unveiled a quick cover case for the likely metal phone earlier this week.
A LG spokesman declined to comment.
Samsung launches Reserve $100 credit promotion ahead of Unpacked event
Samsung just showed off this 200-megapixel smartphone image sensor
Samsung Display brings advanced smartphone OLED touch tech to notebooks

redpajama_set_name: RedPajamaCommonCrawl


At Lawson Health Research Institute, we value investment in research.
We develop or contribute a number of reports to illustrate our strategic directions and our progress as we work to further scientific knowledge to advance health care around the world.
Lawson Link magazine showcases the quality and impact of Lawson research, the breadth of areas where we are making a difference in patient care, and the contributions we are making to London's knowledge economy.
Readers will also learn about the different aspects of a research enterprise, such as the translation of discovery from the lab to clinical trials and eventually patient care, as well as facilities and technology expertise.
These stories are proof that the discovery of innovative new treatments leads to improved health outcomes and a higher quality of life for patients, highlighting the importance of investment in health research.
Read the latest issue of Lawson Link.
The Lawson Strategic Plan 2014-2018 was crafted by our scientific leaders, refined with input from all Lawson investigators. The plan is aligned with the strategic priorities of our university partner, Western University, and our two sponsoring hospitals, London Health Sciences Centre and St. Joseph's Health Care London.
The research directions articulated in the Lawson Strategic Plan 2014-2018 emphasize our strategic capacity and capabilities built on a strong foundation of translatable health science. The overall objective is to deliver disruptive innovation with the potential to positively impact the health of individuals globally.
Over the past two years, we have made incredible strides towards implementing Lawson's strategic plan. We are now pleased to share our 2015 Progress Report, which highlights our achievements.


text:
Q: Loop through and combine schemas/tables with UNION_ALL I want to combine tables from different schemas using UNION_ALL. The tables have the same schema, like in this toy example:
class1.names
+----------+
| id | name|
+----------+
| 1  | jon |
| 2  | ann |
| 3  | rob |

class2.names
+----------+
| id | name|
+----------+
| 1  | rav |
| 2  | meg |
| 3  | mat |

I can hardcode the list of classes into an array, or more preferably, obtain them using a query like so:
SELECT DISTINCT(TABLE_SCHEMA)
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_TYPE = 'BASE TABLE'

I want to combine the tables like so:
SELECT *, 'class1' as class FROM class1.names
UNION_ALL
SELECT *, 'class2' as class FROM class2.names
UNION_ALL
etc.

But there will be a lot more going on in the schema-level queries than SELECT *, 'class1'... so I want to do this using a loop or some other systematic method.
I'm looking at dynamic sql or using GROUP_CONCAT with 'UNION_ALL' as a separator, but I'm having trouble making progress.
Addendum: I know this is poor schema design but I can't do anything about that right now.

A: In Snowflake, this dynamic SQL :
with a as (
  select * from information_schema.tables
  where table_catalog like 'CLASS%' and table_name = 'NAMES' and table_type = 'BASE TABLE'
),

b as (
  select *,
    $$SELECT *, 'SCHEMA' as class FROM SCHEMA.names$$ as t,
    replace(t,'SCHEMA',lower(table_schema)) as sql,
  from a
)

select listagg(sql,'\nUNION ALL\n') within group (order by table_schema, table_catalog)
from b;

will produce :
SELECT *, 'class1' as class FROM class1.names
UNION ALL
SELECT *, 'class2' as class FROM class2.names
UNION ALL
etc.

The $$...$$ are an alternative to single quotes for string literals.  You can also escape single quotes by doubling them up.

A: If i understand your correctly:
select listagg('select ' || f.value, ' union all ') from table(flatten(input => parse_json(
                                                    '[1, ,77]'))) f;
+----------------------------------------------+
| LISTAGG('SELECT ' || F.VALUE, ' UNION ALL ') |
|----------------------------------------------|
| select 1 union all select 77                 |
+----------------------------------------------+
1 Row(s) produced. Time Elapsed: 0.739s

and than:
select 1 union all select 77;
+----+
|  1 |
|----|
|  1 |
| 77 |
+----+
2 Row(s) produced. Time Elapsed: 0.638s

A: Maybe something like:
SELECT DISTINCT case when row_number() OVER (ORDER BY lower(table_schema) || '.' || lower(table_name)) > 1 then 'UNION ALL ' else '' end ||
                   'SELECT *, ''' || lower(table_schema) || ''' AS class FROM ' || lower(table_schema) || '.' || lower(table_name) || '' union_stmt
FROM INFORMATION_SCHEMA.TABLES
WHERE table_type = 'BASE TABLE'
and table_name like 'HPSA%';