text:
<!DOCTYPE html>

<TITLE>Manpage of MAKEDUMPFILE</TITLE>
<meta charset="utf-8">
<link rel="stylesheet" href="/css/main.css" type="text/css">

linux manpages

{"type":"documentation"}

MAKEDUMPFILE

Section: Linux System Administrator's Manual (8)
Updated: 11 November 2009
Index
Return to Main Contents

NAME

makedumpfile - make a small dumpfile of kdump

SYNOPSIS

makedumpfile    [OPTION] [-x VMLINUX|-i VMCOREINFO] VMCORE DUMPFILE

makedumpfile -F [OPTION] [-x VMLINUX|-i VMCOREINFO] VMCORE

makedumpfile   [OPTION] -x VMLINUX --config FILTERCONFIGFILE VMCORE DUMPFILE

makedumpfile -R DUMPFILE

makedumpfile --split [OPTION] [-x VMLINUX|-i VMCOREINFO] VMCORE DUMPFILE1 DUMPFILE2 [DUMPFILE3 ..]

makedumpfile --reassemble DUMPFILE1 DUMPFILE2 [DUMPFILE3 ..] DUMPFILE

makedumpfile -g VMCOREINFO -x VMLINUX

makedumpfile -E [--xen-syms XEN-SYMS|--xen-vmcoreinfo VMCOREINFO] VMCORE DUMPFILE

makedumpfile --dump-dmesg [-x VMLINUX|-i VMCOREINFO] VMCORE LOGFILE

makedumpfile    [OPTION] -x VMLINUX --diskset=VMCORE1 --diskset=VMCORE2 [--diskset=VMCORE3 ..] DUMPFILE

makedumpfile

-h

makedumpfile

-v

DESCRIPTION

With kdump, the memory image of the first kernel (called "panicked kernel") can
be taken as /proc/vmcore while the second kernel (called "kdump kernel" or
"capture kernel") is running. This document represents /proc/vmcore as
VMCORE. makedumpfile makes a small DUMPFILE by compressing dump
data or by excluding unnecessary pages for analysis, or both. makedumpfile
needs the first kernel's debug information, so that it can distinguish
unnecessary pages by analyzing how the first kernel uses the memory.
The information can be taken from VMLINUX or VMCOREINFO.

makedumpfile can exclude the following types of pages while copying
VMCORE to DUMPFILE, and a user can choose which type of pages will
be excluded.

- Pages filled with zero

- Cache pages

- User process data pages

- Free pages

makedumpfile provides two DUMPFILE formats (the ELF format and the
kdump-compressed format). By default, makedumpfile makes a DUMPFILE in
the kdump-compressed format. The kdump-compressed format is readable only with
the crash utility, and it can be smaller than the ELF format because of the
compression support. The ELF format is readable with GDB and the crash utility.
If a user wants to use GDB, DUMPFILE format has to be explicitly
specified to be the ELF format.

Apart from the exclusion of unnecessary pages mentioned above, makedumpfile
allows user to filter out targeted kernel data. The filter config file can
be used to specify kernel/module symbols and its members that need to be
filtered out through the erase command syntax. makedumpfile reads the filter
config and builds the list of memory addresses and its sizes after processing
filter commands. The memory locations that require to be filtered out are
then poisoned with character 'X' (58 in Hex). Refer to
makedumpfile.conf(8) for file format.

To analyze the first kernel's memory usage, makedumpfile can refer to
VMCOREINFO instead of VMLINUX. VMCOREINFO contains the first
kernel's information (structure size, field offset, etc.), and VMCOREINFO
is small enough to be included into the second kernel's initrd.

If the second kernel is running on its initrd without mounting a root file
system, makedumpfile cannot refer to VMLINUX because the second kernel's
initrd cannot include a large file like VMLINUX. To solve the problem,
makedumpfile makes VMCOREINFO beforehand, and it refers to
VMCOREINFO instead of VMLINUX while the second kernel is running.

VMCORE has contained VMCOREINFO since linux-2.6.24, and a user does
not need to specify neither -x nor -i option.

If the second kernel is running on its initrd without mounting any file system,
a user needs to transport the dump data to a remote host. To transport the dump
data by SSH, makedumpfile outputs the dump data in the intermediate format (the
flattened format) to the standard output. By piping the output data to SSH,
a user can transport the dump data to a remote host. Note that analysis tools
cannot read the flattened format directly, so on a remote host the received
data in the flattened format needs to be rearranged to a readable
DUMPFILE format by makedumpfile (or makedumpfile-R.pl).

makedumpfile can read a DUMPFILE in the kdump-compressed format instead
of VMCORE and re-filter it. This feature is useful in situation that
users need to reduce the file size of DUMPFILE for sending it somewhere
by ftp/scp/etc. (If all of the page types, which are specified by a new dump_level,
are excluded from an original DUMPFILE already, a new DUMPFILE is the
same as an original DUMPFILE.)

For example, makedumpfile can create a DUMPFILE of dump_level 31 from the
one of dump_level 3 like the following:

Example:

# makedumpfile -c -d 3 /proc/vmcore dumpfile.1

# makedumpfile -c -d 31 dumpfile.1 dumpfile.2

makedumpfile can read VMCORE(s) in three kinds of sadump
formats: single partition format, diskset format and media backup
format, and can convert each of them into kdump-compressed format with
filtering and compression processing. Note that for VMCORE(s)
created by sadump, you always need to pass VMLINUX with -x
option. Also, to pass multiple VMCOREs created on diskset
configuration, you need to use --diskset option.

OPTIONS

-c

Compress dump data by each page.

A user cannot specify this option with -E option, because the ELF format does
not support compressed data.

Example:

# makedumpfile -c -d 31 -x vmlinux /proc/vmcore dumpfile

-d dump_level

Specify the type of unnecessary page for analysis.

Pages of the specified type are not copied to DUMPFILE. The page type
marked in the following table is excluded. A user can specify multiple page
types by setting the sum of each page type for dump_level. The maximum of
dump_level is 31. Note that a dump_level for Xen dump filtering is 0 or 1 on
a machine other than x86_64 (On an x86_64 machine, it is possible to specify
2 or bigger as a dump_level).

If specifying multiple dump_levels with the delimiter ',', makedumpfile retries
to create a DUMPFILE by other dump_level when "No space on device" error
happens. For example, if dump_level is "11,31" and makedumpfile fails
by dump_level 11, makedumpfile retries it by dump_level 31.

Example:

# makedumpfile -d 11 -x vmlinux /proc/vmcore dumpfile

# makedumpfile -d 11,31 -x vmlinux /proc/vmcore dumpfile

  dump | zero | cache|cache  | user | free

 level | page | page |private| data | page

-------+------+------+-------+------+------

     0 |      |      |       |      |

     1 |  X   |      |       |      |

     2 |      |  X   |       |      |

     3 |  X   |  X   |       |      |

     4 |      |  X   |  X    |      |

     5 |  X   |  X   |  X    |      |

     6 |      |  X   |  X    |      |

     7 |  X   |  X   |  X    |      |

     8 |      |      |       |  X   |

     9 |  X   |      |       |  X   |

    10 |      |  X   |       |  X   |

    11 |  X   |  X   |       |  X   |

    12 |      |  X   |  X    |  X   |

    13 |  X   |  X   |  X    |  X   |

    14 |      |  X   |  X    |  X   |

    15 |  X   |  X   |  X    |  X   |

    16 |      |      |       |      |  X

    17 |  X   |      |       |      |  X

    18 |      |  X   |       |      |  X

    19 |  X   |  X   |       |      |  X

    20 |      |  X   |  X    |      |  X

    21 |  X   |  X   |  X    |      |  X

    22 |      |  X   |  X    |      |  X

    23 |  X   |  X   |  X    |      |  X

    24 |      |      |       |  X   |  X

    25 |  X   |      |       |  X   |  X

    26 |      |  X   |       |  X   |  X

    27 |  X   |  X   |       |  X   |  X

    28 |      |  X   |  X    |  X   |  X

    29 |  X   |  X   |  X    |  X   |  X

    30 |      |  X   |  X    |  X   |  X

    31 |  X   |  X   |  X    |  X   |  X

-E

Create DUMPFILE in the ELF format.

This option cannot be specified with -c option, because the ELF format does not
support compressed data.

Example:

# makedumpfile -E -d 31 -x vmlinux /proc/vmcore dumpfile

-f

Force existing DUMPFILE to be overwritten.

Example:

# makedumpfile -f -d 31 -x vmlinux /proc/vmcore dumpfile

This command overwrites DUMPFILE even if it already exists.

-x VMLINUX

Specify the first kernel's VMLINUX with debug information to analyze the
first kernel's memory usage.

This option is necessary if VMCORE does not contain VMCOREINFO,
[-i VMCOREINFO] is not specified, and dump_level is 2 or more.

The page size of the first kernel and the second kernel should match.

Example:

# makedumpfile -d 31 -x vmlinux /proc/vmcore dumpfile

-i VMCOREINFO

Specify VMCOREINFO instead of VMLINUX for analyzing the first kernel's memory usage.

VMCOREINFO should be made beforehand by makedumpfile with -g option, and
it contains the first kernel's information.

This option is necessary if VMCORE does not contain VMCOREINFO,
[-x VMLINUX] is not specified, and dump_level is 2 or more.

Example:

# makedumpfile -d 31 -i vmcoreinfo /proc/vmcore dumpfile

-g VMCOREINFO

Generate VMCOREINFO from the first kernel's VMLINUX with debug
information.

VMCOREINFO must be generated on the system that is running the first
kernel. With -i option, a user can specify VMCOREINFO generated on the
other system that is running the same first kernel. [-x VMLINUX] must be
specified.

Example:

# makedumpfile -g vmcoreinfo -x vmlinux

--config FILTERCONFIGFILE

Used in conjunction with -x VMLINUX option, to specify the filter
config file FILTERCONFIGFILE that contains erase commands to filter out
desired kernel data from vmcore while creating DUMPFILE. For filter
command syntax please refer to makedumpfile.conf(8).

-F

Output the dump data in the flattened format to the standard output for
transporting the dump data by SSH.

Analysis tools cannot read the flattened format directly. For analysis, the
dump data in the flattened format should be rearranged to a normal
DUMPFILE (readable with analysis tools) by -R option. By which option is
specified with -F option, the format of the rearranged DUMPFILE is fixed.
In other words, it is impossible to specify the DUMPFILE format when the
dump data is rearranged with -R option. If specifying -E option with -F option,
the format of the rearranged DUMPFILE is the ELF format. Otherwise, it
is the kdump-compressed format. All the messages are output to standard error
output by -F option because standard output is used for the dump data.

Example:

# makedumpfile -F -c -d 31 -x vmlinux /proc/vmcore \

| ssh user@host "cat > dumpfile.tmp"

# makedumpfile -F -c -d 31 -x vmlinux /proc/vmcore \

| ssh user@host "makedumpfile -R dumpfile"

# makedumpfile -F -E -d 31 -i vmcoreinfo  /proc/vmcore \

| ssh user@host "makedumpfile -R dumpfile"

# makedumpfile -F -E --xen-vmcoreinfo VMCOREINFO /proc/vmcore \

| ssh user@host "makedumpfile -R dumpfile"

-R

Rearrange the dump data in the flattened format from the standard input to a
normal DUMPFILE (readable with analysis tools).

Example:

# makedumpfile -R dumpfile < dumpfile.tmp

# makedumpfile -F -d 31 -x vmlinux /proc/vmcore \

| ssh user@host "makedumpfile -R dumpfile"

Instead of using -R option, a perl script "makedumpfile-R.pl" rearranges the
dump data in the flattened format to a normal DUMPFILE, too. The perl
script does not depend on architecture, and most systems have perl command.
Even if a remote host does not have makedumpfile, it is possible to rearrange
the dump data in the flattened format to a readable DUMPFILE on a remote
host by running this script.

Example:

# makedumpfile -F -d 31 -x vmlinux /proc/vmcore \

| ssh user@host "makedumpfile-R.pl dumpfile"

--split

Split the dump data to multiple DUMPFILEs in parallel. If specifying
DUMPFILEs on different storage devices, a device can share I/O load
with other devices and it reduces time for saving the dump data. The file
size of each DUMPFILE is smaller than the system memory size which
is divided by the number of DUMPFILEs. This feature supports only
the kdump-compressed format.

Example:

# makedumpfile --split -d 31 -x vmlinux /proc/vmcore dumpfile1 dumpfile2

--reassemble

Reassemble multiple DUMPFILEs, which are created by --split option,
into one DUMPFILE. dumpfile1 and dumpfile2 are reassembled into dumpfile
on the following example.

Example:

# makedumpfile --reassemble dumpfile1 dumpfile2 dumpfile

--xen-syms XEN-SYMS

Specify the XEN-SYMS with debug information to analyze the xen's memory usage.
This option extracts the part of xen and domain-0.
-E option must be specified with this option.

Example:

# makedumpfile -E --xen-syms xen-syms /proc/vmcore dumpfile

--xen-vmcoreinfo VMCOREINFO

Specify VMCOREINFO instead of XEN-SYMS for analyzing the xen's memory usage.

VMCOREINFO should be made beforehand by makedumpfile with -g option, and
it contains the xen's information.
-E option must be specified with this option.

Example:

# makedumpfile -E --xen-vmcoreinfo VMCOREINFO /proc/vmcore dumpfile

-X

Exclude all the user domain pages from Xen kdump's VMCORE, and extracts the
part of xen and domain-0. If VMCORE contains VMCOREINFO for Xen, it is
not necessary to specify --xen-syms and --xen-vmcoreinfo.
-E option must be specified with this option.

Example:

# makedumpfile -E -X /proc/vmcore dumpfile

--xen_phys_start xen_phys_start_address

This option is only for x86_64.
Specify the xen_phys_start_address, if the xen code/data is relocatable
and VMCORE does not contain xen_phys_start_address in the CRASHINFO.
xen_phys_start_address can be taken from the line of "Hypervisor code
and data" in /proc/iomem. For example, specify 0xcee00000 as xen_phys_start_address
if /proc/iomem is the following:

  -------------------------------------------------------

  # cat /proc/iomem

  ...

    cee00000-cfd99999 : Hypervisor code and data

  ...

  -------------------------------------------------------

Example:

# makedumpfile -E -X --xen_phys_start 0xcee00000 /proc/vmcore dumpfile

--message-level message_level

Specify the message types.

Users can restrict outputs printed by specifying message_level
with this option. The message type marked with an X in the following
table is printed. For example, according to the table, specifying 7 as
message_level means progress indicator, common message, and error
message are printed, and this is a default value. Note that the maximum
value of message_level is 31.

 message | progress | common  | error   | debug   | report

 level   | indicator| message | message | message | message

---------+----------+---------+---------+---------+---------

       0 |          |         |         |         |

       1 |    X     |         |         |         |

       2 |          |    X    |         |         |

       3 |    X     |    X    |         |         |

       4 |          |         |    X    |         |

       5 |    X     |         |    X    |         |

       6 |          |    X    |    X    |         |

     * 7 |    X     |    X    |    X    |         |

       8 |          |         |         |    X    |

       9 |    X     |         |         |    X    |

      10 |          |    X    |         |    X    |

      11 |    X     |    X    |         |    X    |

      12 |          |         |    X    |    X    |

      13 |    X     |         |    X    |    X    |

      14 |          |    X    |    X    |    X    |

      15 |    X     |    X    |    X    |    X    |

      16 |          |         |         |         |    X

      17 |    X     |         |         |         |    X

      18 |          |    X    |         |         |    X

      19 |    X     |    X    |         |         |    X

      20 |          |         |    X    |         |    X

      21 |    X     |         |    X    |         |    X

      22 |          |    X    |    X    |         |    X

      23 |    X     |    X    |    X    |         |    X

      24 |          |         |         |    X    |    X

      25 |    X     |         |         |    X    |    X

      26 |          |    X    |         |    X    |    X

      27 |    X     |    X    |         |    X    |    X

      28 |          |         |    X    |    X    |    X

      29 |    X     |         |    X    |    X    |    X

      30 |          |    X    |    X    |    X    |    X

      31 |    X     |    X    |    X    |    X    |    X

--vtop virtual_address

This option is useful, when user debugs the translation problem
of virtual address. If specifing virtual_address, its physical
address is printed. It makes debugging easy by comparing the
output of this option with the one of "vtop" subcommand of the
crash utility.
"--vtop" option only prints the translation output, and it does
not affect the dumpfile creation.

--dump-dmesg

This option overrides the normal behavior of makedumpfile.  Instead of
compressing and filtering a VMCORE to make it smaller, it simply
extracts the dmesg log from a VMCORE and writes it to the specified
LOGFILE. If a VMCORE does not contain VMCOREINFO for dmesg,
it is necessary to specfiy [-x VMLINUX] or [-i VMCOREINFO].

Example:

# makedumpfile --dump-dmesg /proc/vmcore dmesgfile

# makedumpfile --dump-dmesg -x vmlinux /proc/vmcore dmesgfile

--diskset=VMCORE

Specify multiple VMCOREs created on sadump diskset configuration
the same number of times as the number of VMCOREs in increasing
order from left to right.  VMCOREs are assembled into a single
DUMPFILE.

Example:

# makedumpfile -x vmlinux --diskset=vmcore1 --diskset=vmcore2 dumpfile

-D

Print debugging message.

-b <order>

cache 2^order pages in ram when generating vmcore info before writing to output

-h

Show help message.

-v

Show the version of makedumpfile.

DIAGNOSTICS

makedumpfile exits with the following value.

0 : makedumpfile succeeded.

1 : makedumpfile failed without the following reasons.

2 : makedumpfile failed due to the different version between  VMLINUX and VMCORE.

3 : makedumpfile failed due to the analysis error of the memory.

AUTHORS

Written by Masaki Tachibana, and Ken'ichi Ohmichi.

SEE ALSO

crash(8), gdb(1), kexec(8), makedumpfile.conf(8)

Index

NAME

SYNOPSIS

DESCRIPTION

OPTIONS

DIAGNOSTICS

AUTHORS

SEE ALSO

This document was created by
man2html,
using the manual pages.

Time: 05:34:27 GMT, December 24, 2015


text:
ACCEPTED

#### According to
Index Fungorum

#### Published in
Mycoscience 44(6): 459 (2003)

#### Original name
Heterobasidion pahangense Corner

### Remarks
null

redpajama_set_name: RedPajamaGithub


text:
=begin
Copyright (c) 2013 ExactTarget, Inc.

All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the

following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this list of conditions and the

following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the

following disclaimer in the documentation and/or other materials provided with the distribution.

3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote

products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,

INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE

DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,

SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR

SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,

WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE

USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

=end

require 'securerandom'
module FuelSDK
	class Response
		# not doing accessor so user, can't update these values from response.
		# You will see in the code some of these
		# items are being updated via back doors and such.
		attr_reader :code, :message, :results, :request_id, :body, :raw

		# some defaults
		def success
			@success ||= false
		end
		alias :success? :success
		alias :status :success # backward compatibility

		def more
			@more ||= false
		end
		alias :more? :more

		def initialize raw, client
			@client = client # keep connection with client in case we request more
			@results = []
			@raw = raw
			unpack raw
		rescue => ex # all else fails return raw
			puts ex.message
			raw
		end

		def continue
			raise NotImplementedError
		end

		private
		def unpack raw
		raise NotImplementedError
		end
	end

	class Client
	attr_accessor :debug, :access_token, :auth_token, :internal_token, :refresh_token,
		:id, :secret, :signature, :package_name, :package_folders, :parent_folders, :auth_token_expiration

	include FuelSDK::Soap
	include FuelSDK::Rest

		def jwt= encoded_jwt
			raise 'Require app signature to decode JWT' unless self.signature
			decoded_jwt, header = JWT.decode(encoded_jwt, self.signature, true)

			self.auth_token = decoded_jwt['request']['user']['oauthToken']
			self.internal_token = decoded_jwt['request']['user']['internalOauthToken']
			self.refresh_token = decoded_jwt['request']['user']['refreshToken']
			self.auth_token_expiration = Time.new + decoded_jwt['request']['user']['expiresIn']
			self.package_name = decoded_jwt['request']['application']['package']
		end

		def initialize(params={}, debug=false)
			self.debug = debug
			client_config = params['client']
			if client_config
			self.id = client_config["id"]
			self.secret = client_config["secret"]
			self.signature = client_config["signature"]
			end

			self.jwt = params['jwt'] if params['jwt']
			self.refresh_token = params['refresh_token'] if params['refresh_token']

			self.wsdl = params["defaultwsdl"] if params["defaultwsdl"]
		end

		def refresh force=false
			raise 'Require Client Id and Client Secret to refresh tokens' unless (id && secret)
			#If we don't already have a token or the token expires within 5 min(300 seconds)
			if (self.access_token.nil? || Time.new + 300 > self.auth_token_expiration || force) then
			payload = Hash.new.tap do |h|
				h['clientId']= id
				h['clientSecret'] = secret
				h['refreshToken'] = refresh_token if refresh_token
				h['accessType'] = 'offline'
			end

			options = Hash.new.tap do |h|
				h['data'] = payload
				h['content_type'] = 'application/json'
				h['params'] = {'legacy' => 1}
			end
			response = post("https://auth.exacttargetapis.com/v1/requestToken", options)
			raise "Unable to refresh token: #{response['message']}" unless response.has_key?('accessToken')

			self.access_token = response['accessToken']
			self.internal_token = response['legacyToken']
			self.auth_token_expiration = Time.new + response['expiresIn']
			self.refresh_token = response['refreshToken'] if response.has_key?("refreshToken")
			return true
			else
			return false
			end
		end

		def refresh!
			refresh true
		end

		def AddSubscriberToList(email, ids, subscriber_key = nil)
			s = FuelSDK::Subscriber.new
			s.client = self
			lists = ids.collect{|id| {'ID' => id}}
			s.properties = {"EmailAddress" => email, "Lists" => lists}
			p s.properties
			s.properties['SubscriberKey'] = subscriber_key if subscriber_key

			# Try to add the subscriber
			if(rsp = s.post and rsp.results.first[:error_code] == '12014')
			# subscriber already exists we need to update.
			rsp = s.patch
			end
			rsp
		end

		def CreateDataExtensions(definitions)
			de = FuelSDK::DataExtension.new
			de.client = self
			de.properties = definitions
			de.post
		end
		def SendTriggeredSends(arrayOfTriggeredRecords)
			sendTS = ET_TriggeredSend.new
			sendTS.authStub = self

			sendTS.properties = arrayOfTriggeredRecords
			sendResponse = sendTS.send

			return sendResponse
		end
		def SendEmailToList(emailID, listID, sendClassficationCustomerKey)
			email = ET_Email::SendDefinition.new
			email.properties = {"Name"=>SecureRandom.uuid, "CustomerKey"=>SecureRandom.uuid, "Description"=>"Created with RubySDK"}
			email.properties["SendClassification"] = {"CustomerKey"=>sendClassficationCustomerKey}
			email.properties["SendDefinitionList"] = {"List"=> {"ID"=>listID}, "DataSourceTypeID"=>"List"}
			email.properties["Email"] = {"ID"=>emailID}
			email.authStub = self
			result = email.post
			if result.status then
				sendresult = email.send
				if sendresult.status then
					deleteresult = email.delete
					return sendresult
				else
					raise "Unable to send using send definition due to: #{result.results[0][:status_message]}"
				end
			else
				raise "Unable to create send definition due to: #{result.results[0][:status_message]}"
			end
		end

		def SendEmailToDataExtension(emailID, sendableDataExtensionCustomerKey, sendClassficationCustomerKey)
			email = ET_Email::SendDefinition.new
			email.properties = {"Name"=>SecureRandom.uuid, "CustomerKey"=>SecureRandom.uuid, "Description"=>"Created with RubySDK"}
			email.properties["SendClassification"] = {"CustomerKey"=> sendClassficationCustomerKey}
			email.properties["SendDefinitionList"] = {"CustomerKey"=> sendableDataExtensionCustomerKey, "DataSourceTypeID"=>"CustomObject"}
			email.properties["Email"] = {"ID"=>emailID}
			email.authStub = self
			result = email.post
			if result.status then
				sendresult = email.send
				if sendresult.status then
					deleteresult = email.delete
					return sendresult
				else
					raise "Unable to send using send definition due to: #{result.results[0][:status_message]}"
				end
			else
				raise "Unable to create send definition due to: #{result.results[0][:status_message]}"
			end
		end
		def CreateAndStartListImport(listId,fileName)
			import = ET_Import.new
			import.authStub = self
			import.properties = {"Name"=> "SDK Generated Import #{DateTime.now.to_s}"}
			import.properties["CustomerKey"] = SecureRandom.uuid
			import.properties["Description"] = "SDK Generated Import"
			import.properties["AllowErrors"] = "true"
			import.properties["DestinationObject"] = {"ID"=>listId}
			import.properties["FieldMappingType"] = "InferFromColumnHeadings"
			import.properties["FileSpec"] = fileName
			import.properties["FileType"] = "CSV"
			import.properties["RetrieveFileTransferLocation"] = {"CustomerKey"=>"ExactTarget Enhanced FTP"}
			import.properties["UpdateType"] = "AddAndUpdate"
			result = import.post

			if result.status then
				return import.start
			else
				raise "Unable to create import definition due to: #{result.results[0][:status_message]}"
			end
		end

		def CreateAndStartDataExtensionImport(dataExtensionCustomerKey, fileName, overwrite)
			import = ET_Import.new
			import.authStub = self
			import.properties = {"Name"=> "SDK Generated Import #{DateTime.now.to_s}"}
			import.properties["CustomerKey"] = SecureRandom.uuid
			import.properties["Description"] = "SDK Generated Import"
			import.properties["AllowErrors"] = "true"
			import.properties["DestinationObject"] = {"ObjectID"=>dataExtensionCustomerKey}
			import.properties["FieldMappingType"] = "InferFromColumnHeadings"
			import.properties["FileSpec"] = fileName
			import.properties["FileType"] = "CSV"
			import.properties["RetrieveFileTransferLocation"] = {"CustomerKey"=>"ExactTarget Enhanced FTP"}
			if overwrite then
				import.properties["UpdateType"] = "Overwrite"
			else
				import.properties["UpdateType"] = "AddAndUpdate"
			end
			result = import.post

			if result.status then
				return import.start
			else
				raise "Unable to create import definition due to: #{result.results[0][:status_message]}"
			end
		end

		def CreateProfileAttributes(allAttributes)
			attrs = ET_ProfileAttribute.new
			attrs.authStub = self
			attrs.properties = allAttributes
			return attrs.post
		end

		def CreateContentAreas(arrayOfContentAreas)
			postC = ET_ContentArea.new
			postC.authStub = self
			postC.properties = arrayOfContentAreas
			sendResponse = postC.post
			return sendResponse
		end
	end
end


text:
ngx-bootstrap

Best way to quickly integrate Bootstrap 3 or Bootstrap 4 Components with Angular

## Links

- [Documentation](http://valor-software.com/ngx-bootstrap/)
- [Release Notes](https://github.com/valor-software/ngx-bootstrap/blob/development/CHANGELOG.md)
- [Slack Community](https://ngx-slack.herokuapp.com)
-

## Table of contents
1. [Getting Started](#getting-started)
2. [Installation instructions](#installation-instructions)
3. [Usage & Demo](#usage--demo)
4. [API](#api)
5. [Troubleshooting](#troubleshooting)
6. [Contributing](#contribution)
7. [License](#license)

# Getting Started

ngx-bootstrap contains all core (and not only) Bootstrap components powered by Angular. So you don't need to include original JS components, but we are using markup and css provided by Bootstrap.

# Installation instructions

Install `ngx-bootstrap` from `npm`
```bash
npm install ngx-bootstrap --save
```

You will need bootstrap styles (Bootstrap 3)

```
<!-- index.html -->
<link href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css" rel="stylesheet">
```

Or Bootstrap 4
```
<!--- index.html -->
<link href="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0-beta/css/bootstrap.min.css" rel="stylesheet">
```
To enable bootstrap 4 theme templates in ngx-bootstrap, please read
[here](https://github.com/valor-software/ngx-bootstrap/blob/development/docs/getting-started/bootstrap4.md)

```
<!-- index.html -->
<link href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css" rel="stylesheet">
```

## How to use it with:
 - `angular-cli` please refer to [getting-started-with-ng-cli](https://github.com/valor-software/ngx-bootstrap/tree/development/docs/getting-started/ng-cli.md)
 - `angular-seed` please refer to [getting-started-with-angular-seed](https://github.com/valor-software/ngx-bootstrap/tree/development/docs/getting-started/angular-seed.md)
 - `system.js` (and [angular2 quickstart](https://angular.io/docs/ts/latest/quickstart.html)) please checkout [sample repository](https://github.com/valor-software/angular2-quickstart)
 - `plnkr` sample available [here](http://bit.ly/ngx-bootstrap-plnkr)
 - `AoT using ngc and rollup` please refer to [ngx-bootstrap-with-aot](https://github.com/valor-software/ngx-bootstrap/tree/development/docs/getting-started/aot.md)
 - `Bootstrap 4 and angular-cli` please refer to [using-with-bootstrap-4-and-angular-cli](https://github.com/valor-software/ngx-bootstrap/tree/development/docs/getting-started/bootstrap4.md)

# How to build lib for development

First time
 - clone repository
 - `npm install`
 - `npm run test` it will build the lib and create a link in `node_modules`

 To run bootstrap 3 and 4 demo:
 - `npm run demo.serve` to serve local demo. This is for testing only, without watchers.

 For local development run
 - `npm run build.watch` in first terminal
 - `ng serve` in second

# Usage & Demo

Main source of API documentation and usage scenarios available here:
[http://valor-software.github.io/ngx-bootstrap/](http://valor-software.github.io/ngx-bootstrap/)

# API
Check demo page for API [reference](http://valor-software.github.io/ngx-bootstrap/)

# Troubleshooting

So you are in trouble, where to get help:
- you can search and ask for help at [StackOverflow](https://stackoverflow.com/questions/tagged/ngx-bootstrap)
- you can join [slack](https://ngx-slack.herokuapp.com/) and ask a question
- if none of this helped please [search](https://github.com/valor-software/ngx-bootstrap/issues?utf8=%E2%9C%93&q=is%3Aissue) and only then open new issue

# Contribution

Are very welcome! And remember, contribution is not only PRs and code, but any help with docs or helping other developers to solve issues are very appreciated! Thanks in advance!

## If you need more modules check [here](https://github.com/valor-software/ng2-plans)

Please read central `ngx` modules [readme](https://github.com/valor-software/ng2-plans) for details, plans and approach

## Credits
Crossbrowser testing sponsored by [Browser Stack](https://www.browserstack.com)
[](https://www.browserstack.com)


text:
/* -*- Mode: C++; tab-width: 50; indent-tabs-mode: nil; c-basic-offset: 2 -*- */
/* vim: set sw=2 ts=50 et cin tw=80 : */
/* This Source Code Form is subject to the terms of the Mozilla Public
 * License, v. 2.0. If a copy of the MPL was not distributed with this
 * file, You can obtain one at http://mozilla.org/MPL/2.0/. */

#include "mozilla/MemoryInfoDumper.h"

#include "mozilla/ClearOnShutdown.h"
#include "mozilla/FileUtils.h"
#include "mozilla/Preferences.h"
#include "mozilla/StaticPtr.h"
#include "mozilla/unused.h"
#include "mozilla/dom/ContentParent.h"
#include "mozilla/dom/ContentChild.h"
#include "nsIConsoleService.h"
#include "nsICycleCollectorListener.h"
#include "nsDirectoryServiceDefs.h"
#include "nsGZFileWriter.h"
#include "nsJSEnvironment.h"
#include "nsPrintfCString.h"

#ifdef XP_WIN
#include <process.h>
#define getpid _getpid
#else
#include <unistd.h>
#endif

#ifdef XP_LINUX
#include <errno.h>
#include <fcntl.h>
#include <sys/types.h>
#include <sys/stat.h>
#endif

#ifdef ANDROID
#include "android/log.h"
#endif

#ifdef LOG
#undef LOG
#endif

#ifdef ANDROID
#define LOG(...) __android_log_print(ANDROID_LOG_INFO, "Gecko:MemoryInfoDumper", ## __VA_ARGS__)
#else
#define LOG(...)
#endif

using namespace mozilla;
using namespace mozilla::dom;

namespace mozilla {
namespace {

class DumpMemoryReportsRunnable : public nsRunnable
{
public:
  DumpMemoryReportsRunnable(const nsAString& aIdentifier,
                            bool aMinimizeMemoryUsage,
                            bool aDumpChildProcesses)

      : mIdentifier(aIdentifier)
      , mMinimizeMemoryUsage(aMinimizeMemoryUsage)
      , mDumpChildProcesses(aDumpChildProcesses)
  {}

  NS_IMETHOD Run()
  {
    MemoryInfoDumper::DumpMemoryReportsToFile(
      mIdentifier, mMinimizeMemoryUsage, mDumpChildProcesses);
    return NS_OK;
  }

private:
  const nsString mIdentifier;
  const bool mMinimizeMemoryUsage;
  const bool mDumpChildProcesses;
};

class GCAndCCLogDumpRunnable : public nsRunnable
{
public:
  GCAndCCLogDumpRunnable(const nsAString& aIdentifier,
                         bool aDumpAllTraces,
                         bool aDumpChildProcesses)
    : mIdentifier(aIdentifier)
    , mDumpAllTraces(aDumpAllTraces)
    , mDumpChildProcesses(aDumpChildProcesses)
  {}

  NS_IMETHOD Run()
  {
    MemoryInfoDumper::DumpGCAndCCLogsToFile(
      mIdentifier, mDumpAllTraces, mDumpChildProcesses);
    return NS_OK;
  }

private:
  const nsString mIdentifier;
  const bool mDumpAllTraces;
  const bool mDumpChildProcesses;
};

} // anonymous namespace

#ifdef XP_LINUX // {
namespace {

/*
 * The following code supports dumping about:memory upon receiving a signal.
 *
 * We listen for the following signals:
 *
 *  - SIGRTMIN:     Dump our memory reporters (and those of our child
 *                  processes),
 *  - SIGRTMIN + 1: Dump our memory reporters (and those of our child
 *                  processes) after minimizing memory usage, and
 *  - SIGRTMIN + 2: Dump the GC and CC logs in this and our child processes.
 *
 * When we receive one of these signals, we write the signal number to a pipe.
 * The IO thread then notices that the pipe has been written to, and kicks off
 * the appropriate task on the main thread.
 *
 * This scheme is similar to using signalfd(), except it's portable and it
 * doesn't require the use of sigprocmask, which is problematic because it
 * masks signals received by child processes.
 *
 * In theory, we could use Chromium's MessageLoopForIO::CatchSignal() for this.
 * But that uses libevent, which does not handle the realtime signals (bug
 * 794074).
 */

// It turns out that at least on some systems, SIGRTMIN is not a compile-time
// constant, so these have to be set at runtime.
static int sDumpAboutMemorySignum;         // SIGRTMIN
static int sDumpAboutMemoryAfterMMUSignum; // SIGRTMIN + 1
static int sGCAndCCDumpSignum;             // SIGRTMIN + 2

// This is the write-end of a pipe that we use to notice when a
// dump-about-memory signal occurs.
static int sDumpAboutMemoryPipeWriteFd = -1;

void
DumpAboutMemorySignalHandler(int aSignum)
{
  // This is a signal handler, so everything in here needs to be
  // async-signal-safe.  Be careful!

  if (sDumpAboutMemoryPipeWriteFd != -1) {
    uint8_t signum = static_cast<int>(aSignum);
    write(sDumpAboutMemoryPipeWriteFd, &signum, sizeof(signum));
  }
}

/**
 * Abstract base class for something which watches an fd and takes action when
 * we can read from it without blocking.
 */
class FdWatcher : public MessageLoopForIO::Watcher
                , public nsIObserver
{
protected:
  MessageLoopForIO::FileDescriptorWatcher mReadWatcher;
  int mFd;

public:
  FdWatcher()
    : mFd(-1)
  {
    MOZ_ASSERT(NS_IsMainThread());
  }

  virtual ~FdWatcher()
  {
    // StopWatching should have run.
    MOZ_ASSERT(mFd == -1);
  }

  /**
   * Open the fd to watch.  If we encounter an error, return -1.
   */
  virtual int OpenFd() = 0;

  /**
   * Called when you can read() from the fd without blocking.  Note that this
   * function is also called when you're at eof (read() returns 0 in this case).
   */
  virtual void OnFileCanReadWithoutBlocking(int aFd) = 0;
  virtual void OnFileCanWriteWithoutBlocking(int Afd) {};

  NS_DECL_ISUPPORTS

  /**
   * Initialize this object.  This should be called right after the object is
   * constructed.  (This would go in the constructor, except we interact with
   * XPCOM, which we can't do from a constructor because our refcount is 0 at
   * that point.)
   */
  void Init()
  {
    MOZ_ASSERT(NS_IsMainThread());

    nsCOMPtr<nsIObserverService> os = services::GetObserverService();
    os->AddObserver(this, "xpcom-shutdown", /* ownsWeak = */ false);

    XRE_GetIOMessageLoop()->PostTask(
        FROM_HERE,
        NewRunnableMethod(this, &FdWatcher::StartWatching));
  }

  // Implementations may call this function multiple times if they ensure that
  // it's safe to call OpenFd() multiple times and they call StopWatching()
  // first.
  virtual void StartWatching()
  {
    MOZ_ASSERT(XRE_GetIOMessageLoop() == MessageLoopForIO::current());
    MOZ_ASSERT(mFd == -1);

    mFd = OpenFd();
    if (mFd == -1) {
      LOG("FdWatcher: OpenFd failed.");
      return;
    }

    MessageLoopForIO::current()->WatchFileDescriptor(
      mFd, /* persistent = */ true,
      MessageLoopForIO::WATCH_READ,
      &mReadWatcher, this);
  }

  // Since implementations can call StartWatching() multiple times, they can of
  // course call StopWatching() multiple times.
  virtual void StopWatching()
  {
    MOZ_ASSERT(XRE_GetIOMessageLoop() == MessageLoopForIO::current());

    mReadWatcher.StopWatchingFileDescriptor();
    if (mFd != -1) {
      close(mFd);
      mFd = -1;
    }
  }

  NS_IMETHOD Observe(nsISupports* aSubject, const char* aTopic,
                     const PRUnichar* aData)
  {
    MOZ_ASSERT(NS_IsMainThread());
    MOZ_ASSERT(!strcmp(aTopic, "xpcom-shutdown"));

    XRE_GetIOMessageLoop()->PostTask(
        FROM_HERE,
        NewRunnableMethod(this, &FdWatcher::StopWatching));

    return NS_OK;
  }
};

NS_IMPL_THREADSAFE_ISUPPORTS1(FdWatcher, nsIObserver);

class SignalPipeWatcher : public FdWatcher
{
public:
  static void Create()
  {
    nsRefPtr<SignalPipeWatcher> sw = new SignalPipeWatcher();
    sw->Init();
  }

  virtual ~SignalPipeWatcher()
  {
    MOZ_ASSERT(sDumpAboutMemoryPipeWriteFd == -1);
  }

  virtual int OpenFd()
  {
    MOZ_ASSERT(XRE_GetIOMessageLoop() == MessageLoopForIO::current());

    sDumpAboutMemorySignum = SIGRTMIN;
    sDumpAboutMemoryAfterMMUSignum = SIGRTMIN + 1;
    sGCAndCCDumpSignum = SIGRTMIN + 2;

    // Create a pipe.  When we receive a signal in our signal handler, we'll
    // write the signum to the write-end of this pipe.
    int pipeFds[2];
    if (pipe(pipeFds)) {
      LOG("SignalPipeWatcher failed to create pipe.");
      return -1;
    }

    // Close this pipe on calls to exec().
    fcntl(pipeFds[0], F_SETFD, FD_CLOEXEC);
    fcntl(pipeFds[1], F_SETFD, FD_CLOEXEC);

    int readFd = pipeFds[0];
    sDumpAboutMemoryPipeWriteFd = pipeFds[1];

    struct sigaction action;
    memset(&action, 0, sizeof(action));
    sigemptyset(&action.sa_mask);
    action.sa_handler = DumpAboutMemorySignalHandler;

    if (sigaction(sDumpAboutMemorySignum, &action, nullptr)) {
      LOG("SignalPipeWatcher failed to register about:memory "
          "dump signal handler.");
    }
    if (sigaction(sDumpAboutMemoryAfterMMUSignum, &action, nullptr)) {
      LOG("SignalPipeWatcher failed to register about:memory "
          "dump after MMU signal handler.");
    }
    if (sigaction(sGCAndCCDumpSignum, &action, nullptr)) {
      LOG("Failed to register GC+CC dump signal handler.");
    }

    return readFd;
  }

  virtual void StopWatching()
  {
    MOZ_ASSERT(XRE_GetIOMessageLoop() == MessageLoopForIO::current());

    // Close sDumpAboutMemoryPipeWriteFd /after/ setting the fd to -1.
    // Otherwise we have the (admittedly far-fetched) race where we
    //
    //  1) close sDumpAboutMemoryPipeWriteFd
    //  2) open a new fd with the same number as sDumpAboutMemoryPipeWriteFd
    //     had.
    //  3) receive a signal, then write to the fd.
    int pipeWriteFd = sDumpAboutMemoryPipeWriteFd;
    PR_ATOMIC_SET(&sDumpAboutMemoryPipeWriteFd, -1);
    close(pipeWriteFd);

    FdWatcher::StopWatching();
  }

  virtual void OnFileCanReadWithoutBlocking(int aFd)
  {
    MOZ_ASSERT(XRE_GetIOMessageLoop() == MessageLoopForIO::current());

    uint8_t signum;
    ssize_t numReceived = read(aFd, &signum, sizeof(signum));
    if (numReceived != sizeof(signum)) {
      LOG("Error reading from buffer in "
          "SignalPipeWatcher::OnFileCanReadWithoutBlocking.");
      return;
    }

    if (signum == sDumpAboutMemorySignum ||
        signum == sDumpAboutMemoryAfterMMUSignum) {
      // Dump our memory reports (but run this on the main thread!).
      bool doMMUFirst = signum == sDumpAboutMemoryAfterMMUSignum;
      nsRefPtr<DumpMemoryReportsRunnable> runnable =
        new DumpMemoryReportsRunnable(
            /* identifier = */ EmptyString(),
            doMMUFirst,
            /* dumpChildProcesses = */ true);
      NS_DispatchToMainThread(runnable);
    }
    else if (signum == sGCAndCCDumpSignum) {
      // Dump GC and CC logs (from the main thread).
      nsRefPtr<GCAndCCLogDumpRunnable> runnable =
        new GCAndCCLogDumpRunnable(
            /* identifier = */ EmptyString(),
            /* allTraces = */ true,
            /* dumpChildProcesses = */ true);
      NS_DispatchToMainThread(runnable);
    }
    else {
      LOG("SignalPipeWatcher got unexpected signum.");
    }
  }
};

class FifoWatcher : public FdWatcher
{
public:
  static void MaybeCreate()
  {
    MOZ_ASSERT(NS_IsMainThread());

    if (XRE_GetProcessType() != GeckoProcessType_Default) {
      // We want this to be main-process only, since two processes can't listen
      // to the same fifo.
      return;
    }

    if (!Preferences::GetBool("memory_info_dumper.watch_fifo.enabled", false)) {
      LOG("Fifo watcher disabled via pref.");
      return;
    }

    // The FifoWatcher is held alive by the observer service.
    nsRefPtr<FifoWatcher> fw = new FifoWatcher();
    fw->Init();
  }

  virtual int OpenFd()
  {
    // If the memory_info_dumper.directory pref is specified, put the fifo
    // there.  Otherwise, put it into the system's tmp directory.

    nsCOMPtr<nsIFile> file;
    nsAutoCString dirPath;
    nsresult rv = Preferences::GetCString(
      "memory_info_dumper.watch_fifo.directory", &dirPath);

    if (NS_SUCCEEDED(rv)) {
      rv = XRE_GetFileFromPath(dirPath.get(), getter_AddRefs(file));
      if (NS_FAILED(rv)) {
        LOG("FifoWatcher failed to open file \"%s\"", dirPath.get());
        return -1;
      }
    } else {
      rv = NS_GetSpecialDirectory(NS_OS_TEMP_DIR, getter_AddRefs(file));
      NS_ENSURE_SUCCESS(rv, -1);
    }

    rv = file->AppendNative(NS_LITERAL_CSTRING("debug_info_trigger"));
    NS_ENSURE_SUCCESS(rv, -1);

    nsAutoCString path;
    rv = file->GetNativePath(path);
    NS_ENSURE_SUCCESS(rv, -1);

    // unlink might fail because the file doesn't exist, or for other reasons.
    // But we don't care it fails; any problems will be detected later, when we
    // try to mkfifo or open the file.
    if (unlink(path.get())) {
      LOG("FifoWatcher::OpenFifo unlink failed; errno=%d.  "
          "Continuing despite error.", errno);
    }

    if (mkfifo(path.get(), 0766)) {
      LOG("FifoWatcher::OpenFifo mkfifo failed; errno=%d", errno);
      return -1;
    }

#ifdef ANDROID
    // Android runs with a umask, so we need to chmod our fifo to make it
    // world-writable.
    chmod(path.get(), 0666);
#endif

    int fd;
    do {
      // The fifo will block until someone else has written to it.  In
      // particular, open() will block until someone else has opened it for
      // writing!  We want open() to succeed and read() to block, so we open
      // with NONBLOCK and then fcntl that away.
      fd = open(path.get(), O_RDONLY | O_NONBLOCK);
    } while (fd == -1 && errno == EINTR);

    if (fd == -1) {
      LOG("FifoWatcher::OpenFifo open failed; errno=%d", errno);
      return -1;
    }

    // Make fd blocking now that we've opened it.
    if (fcntl(fd, F_SETFL, 0)) {
      close(fd);
      return -1;
    }

    return fd;
  }

  virtual void OnFileCanReadWithoutBlocking(int aFd)
  {
    MOZ_ASSERT(XRE_GetIOMessageLoop() == MessageLoopForIO::current());

    char buf[1024];
    int nread;
    do {
      // sizeof(buf) - 1 to leave space for the null-terminator.
      nread = read(aFd, buf, sizeof(buf));
    } while(nread == -1 && errno == EINTR);

    if (nread == -1) {
      // We want to avoid getting into a situation where
      // OnFileCanReadWithoutBlocking is called in an infinite loop, so when
      // something goes wrong, stop watching the fifo altogether.
      LOG("FifoWatcher hit an error (%d) and is quitting.", errno);
      StopWatching();
      return;
    }

    if (nread == 0) {
      // If we get EOF, that means that the other side closed the fifo.  We need
      // to close and re-open the fifo; if we don't,
      // OnFileCanWriteWithoutBlocking will be called in an infinite loop.

      LOG("FifoWatcher closing and re-opening fifo.");
      StopWatching();
      StartWatching();
      return;
    }

    nsAutoCString inputStr;
    inputStr.Append(buf, nread);

    // Trimming whitespace is important because if you do
    //   |echo "foo" >> debug_info_trigger|,
    // it'll actually write "foo\n" to the fifo.
    inputStr.Trim("\b\t\r\n");

    bool doMemoryReport = inputStr == NS_LITERAL_CSTRING("memory report");
    bool doMMUMemoryReport = inputStr == NS_LITERAL_CSTRING("minimize memory report");
    bool doAllTracesGCCCDump = inputStr == NS_LITERAL_CSTRING("gc log");
    bool doSmallGCCCDump = inputStr == NS_LITERAL_CSTRING("abbreviated gc log");

    if (doMemoryReport || doMMUMemoryReport) {
      LOG("FifoWatcher dispatching memory report runnable.");
      nsRefPtr<DumpMemoryReportsRunnable> runnable =
        new DumpMemoryReportsRunnable(
          /* identifier = */ EmptyString(),
          doMMUMemoryReport,
          /* dumpChildProcesses = */ true);
      NS_DispatchToMainThread(runnable);
    } else if (doAllTracesGCCCDump || doSmallGCCCDump) {
      LOG("FifoWatcher dispatching GC/CC log runnable.");
      nsRefPtr<GCAndCCLogDumpRunnable> runnable =
        new GCAndCCLogDumpRunnable(
            /* identifier = */ EmptyString(),
            doAllTracesGCCCDump,
            /* dumpChildProcesses = */ true);
      NS_DispatchToMainThread(runnable);
    } else {
      LOG("Got unexpected value from fifo; ignoring it.");
    }
  }
};

} // anonymous namespace
#endif // } XP_LINUX

/* static */ void
MemoryInfoDumper::Initialize()
{
#ifdef XP_LINUX
  SignalPipeWatcher::Create();
  FifoWatcher::MaybeCreate();
#endif
}

static void
EnsureNonEmptyIdentifier(nsAString& aIdentifier)
{
  if (!aIdentifier.IsEmpty()) {
    return;
  }

  // If the identifier is empty, set it to the number of whole seconds since the
  // epoch.  This identifier will appear in the files that this process
  // generates and also the files generated by this process's children, allowing
  // us to identify which files are from the same memory report request.
  aIdentifier.AppendInt(static_cast<int64_t>(PR_Now()) / 1000000);
}

/* static */ void
MemoryInfoDumper::DumpMemoryReportsToFile(
    const nsAString& aIdentifier,
    bool aMinimizeMemoryUsage,
    bool aDumpChildProcesses)
{
  nsString identifier(aIdentifier);
  EnsureNonEmptyIdentifier(identifier);

  // Kick off memory report dumps in our child processes, if applicable.  We
  // do this before doing our own report because writing a report may be I/O
  // bound, in which case we want to busy the CPU with other reports while we
  // work on our own.
  if (aDumpChildProcesses) {
    nsTArray<ContentParent*> children;
    ContentParent::GetAll(children);
    for (uint32_t i = 0; i < children.Length(); i++) {
      unused << children[i]->SendDumpMemoryReportsToFile(
          identifier, aMinimizeMemoryUsage, aDumpChildProcesses);
    }
  }

  if (aMinimizeMemoryUsage) {
    // Minimize memory usage, then run DumpMemoryReportsToFile again.
    nsRefPtr<DumpMemoryReportsRunnable> callback =
      new DumpMemoryReportsRunnable(identifier,
          /* minimizeMemoryUsage = */ false,
          /* dumpChildProcesses = */ false);
    nsCOMPtr<nsIMemoryReporterManager> mgr =
      do_GetService("@mozilla.org/memory-reporter-manager;1");
    NS_ENSURE_TRUE(mgr,);
    nsCOMPtr<nsICancelableRunnable> runnable;
    mgr->MinimizeMemoryUsage(callback, getter_AddRefs(runnable));
    return;
  }

  DumpMemoryReportsToFileImpl(identifier);
}

/* static */ void
MemoryInfoDumper::DumpGCAndCCLogsToFile(
  const nsAString& aIdentifier,
  bool aDumpAllTraces,
  bool aDumpChildProcesses)
{
  nsString identifier(aIdentifier);
  EnsureNonEmptyIdentifier(identifier);

  if (aDumpChildProcesses) {
    nsTArray<ContentParent*> children;
    ContentParent::GetAll(children);
    for (uint32_t i = 0; i < children.Length(); i++) {
      unused << children[i]->SendDumpGCAndCCLogsToFile(
          identifier, aDumpAllTraces, aDumpChildProcesses);
    }
  }

  nsCOMPtr<nsICycleCollectorListener> logger =
    do_CreateInstance("@mozilla.org/cycle-collector-logger;1");
  logger->SetFilenameIdentifier(identifier);

  if (aDumpAllTraces) {
    nsCOMPtr<nsICycleCollectorListener> allTracesLogger;
    logger->AllTraces(getter_AddRefs(allTracesLogger));
    logger = allTracesLogger;
  }

  nsJSContext::CycleCollectNow(logger);
}

#define DUMP(o, s) \
  do { \
    nsresult rv = (o)->Write(s); \
    NS_ENSURE_SUCCESS(rv, rv); \
  } while (0)

static nsresult
DumpReport(nsIGZFileWriter *aWriter, bool aIsFirst,
  const nsACString &aProcess, const nsACString &aPath, int32_t aKind,
  int32_t aUnits, int64_t aAmount, const nsACString &aDescription)
{
  DUMP(aWriter, aIsFirst ? "[" : ",");

  // We only want to dump reports for this process.  If |aProcess| is
  // non-NULL that means we've received it from another process in response
  // to a "child-memory-reporter-request" event;  ignore such reports.
  if (!aProcess.IsEmpty()) {
    return NS_OK;
  }

  // Generate the process identifier, which is of the form "$PROCESS_NAME
  // (pid $PID)", or just "(pid $PID)" if we don't have a process name.  If
  // we're the main process, we let $PROCESS_NAME be "Main Process".
  nsAutoCString processId;
  if (XRE_GetProcessType() == GeckoProcessType_Default) {
    // We're the main process.
    processId.AssignLiteral("Main Process ");
  } else if (ContentChild *cc = ContentChild::GetSingleton()) {
    // Try to get the process name from ContentChild.
    nsAutoString processName;
    cc->GetProcessName(processName);
    processId.Assign(NS_ConvertUTF16toUTF8(processName));
    if (!processId.IsEmpty()) {
      processId.AppendLiteral(" ");
    }
  }

  // Add the PID to the identifier.
  unsigned pid = getpid();
  processId.Append(nsPrintfCString("(pid %u)", pid));

  DUMP(aWriter, "\n    {\"process\": \"");
  DUMP(aWriter, processId);

  DUMP(aWriter, "\", \"path\": \"");
  nsCString path(aPath);
  path.ReplaceSubstring("\\", "\\\\");    /* <backslash> --> \\ */
  path.ReplaceSubstring("\"", "\\\"");    // " --> \"
  DUMP(aWriter, path);

  DUMP(aWriter, "\", \"kind\": ");
  DUMP(aWriter, nsPrintfCString("%d", aKind));

  DUMP(aWriter, ", \"units\": ");
  DUMP(aWriter, nsPrintfCString("%d", aUnits));

  DUMP(aWriter, ", \"amount\": ");
  DUMP(aWriter, nsPrintfCString("%lld", aAmount));

  nsCString description(aDescription);
  description.ReplaceSubstring("\\", "\\\\");    /* <backslash> --> \\ */
  description.ReplaceSubstring("\"", "\\\"");    // " --> \"
  description.ReplaceSubstring("\n", "\\n");     // <newline> --> \n
  DUMP(aWriter, ", \"description\": \"");
  DUMP(aWriter, description);
  DUMP(aWriter, "\"}");

  return NS_OK;
}

class DumpMultiReporterCallback MOZ_FINAL : public nsIMemoryMultiReporterCallback
{
  public:
    NS_DECL_ISUPPORTS

      NS_IMETHOD Callback(const nsACString &aProcess, const nsACString &aPath,
          int32_t aKind, int32_t aUnits, int64_t aAmount,
          const nsACString &aDescription,
          nsISupports *aData)
      {
        nsCOMPtr<nsIGZFileWriter> writer = do_QueryInterface(aData);
        NS_ENSURE_TRUE(writer, NS_ERROR_FAILURE);

        // The |isFirst = false| assumes that at least one single reporter is
        // present and so will have been processed in
        // DumpMemoryReportsToFileImpl() below.
        return DumpReport(writer, /* isFirst = */ false, aProcess, aPath,
            aKind, aUnits, aAmount, aDescription);
        return NS_OK;
      }
};

NS_IMPL_ISUPPORTS1(
    DumpMultiReporterCallback
    , nsIMemoryMultiReporterCallback
    )

static void
MakeFilename(const char *aPrefix, const nsAString &aIdentifier,
             const char *aSuffix, nsACString &aResult)
{
  aResult = nsPrintfCString("%s-%s-%d.%s",
                            aPrefix,
                            NS_ConvertUTF16toUTF8(aIdentifier).get(),
                            getpid(), aSuffix);
}

static nsresult
OpenTempFile(const nsACString &aFilename, nsIFile* *aFile)
{
  nsresult rv = NS_GetSpecialDirectory(NS_OS_TEMP_DIR, aFile);
  NS_ENSURE_SUCCESS(rv, rv);

#ifdef ANDROID
  // /data/local/tmp is a true tmp directory; anyone can create a file there,
  // but only the user which created the file can remove it.  We want non-root
  // users to be able to remove these files, so we write them into a
  // subdirectory of the temp directory and chmod 777 that directory.
  rv = (*aFile)->AppendNative(NS_LITERAL_CSTRING("memory-reports"));
  NS_ENSURE_SUCCESS(rv, rv);

  // It's OK if this fails; that probably just means that the directory already
  // exists.
  (*aFile)->Create(nsIFile::DIRECTORY_TYPE, 0777);

  nsAutoCString dirPath;
  rv = (*aFile)->GetNativePath(dirPath);
  NS_ENSURE_SUCCESS(rv, rv);

  while (chmod(dirPath.get(), 0777) == -1 && errno == EINTR) {}
#endif

  nsCOMPtr<nsIFile> file(*aFile);

  rv = file->AppendNative(aFilename);
  NS_ENSURE_SUCCESS(rv, rv);

  rv = file->CreateUnique(nsIFile::NORMAL_FILE_TYPE, 0666);
  NS_ENSURE_SUCCESS(rv, rv);

#ifdef ANDROID
    // Make this file world-read/writable; the permissions passed to the
    // CreateUnique call above are not sufficient on Android, which runs with a
    // umask.
    nsAutoCString path;
    rv = file->GetNativePath(path);
    NS_ENSURE_SUCCESS(rv, rv);

    while (chmod(path.get(), 0666) == -1 && errno == EINTR) {}
#endif

  return NS_OK;
}

#ifdef MOZ_DMD
struct DMDWriteState
{
  static const size_t kBufSize = 4096;
  char mBuf[kBufSize];
  nsRefPtr<nsGZFileWriter> mGZWriter;

  DMDWriteState(nsGZFileWriter *aGZWriter)
    : mGZWriter(aGZWriter)
  {}
};

static void DMDWrite(void* aState, const char* aFmt, va_list ap)
{
  DMDWriteState *state = (DMDWriteState*)aState;
  vsnprintf(state->mBuf, state->kBufSize, aFmt, ap);
  unused << state->mGZWriter->Write(state->mBuf);
}
#endif

/* static */ nsresult
MemoryInfoDumper::DumpMemoryReportsToFileImpl(
  const nsAString& aIdentifier)
{
  MOZ_ASSERT(!aIdentifier.IsEmpty());

  // Open a new file named something like
  //
  //   incomplete-memory-report-<identifier>-<pid>.json.gz
  //
  // in NS_OS_TEMP_DIR for writing.  When we're finished writing the report,
  // we'll rename this file and get rid of the "incomplete-" prefix.
  //
  // We do this because we don't want scripts which poll the filesystem
  // looking for memory report dumps to grab a file before we're finished
  // writing to it.

  // Note that |mrFilename| is missing the "incomplete-" prefix; we'll tack
  // that on in a moment.
  nsCString mrFilename;
  MakeFilename("memory-report", aIdentifier, "json.gz", mrFilename);

  nsCOMPtr<nsIFile> mrTmpFile;
  nsresult rv;
  rv = OpenTempFile(NS_LITERAL_CSTRING("incomplete-") + mrFilename,
                    getter_AddRefs(mrTmpFile));
  NS_ENSURE_SUCCESS(rv, rv);

  nsRefPtr<nsGZFileWriter> writer = new nsGZFileWriter();
  rv = writer->Init(mrTmpFile);
  NS_ENSURE_SUCCESS(rv, rv);

  // Dump the memory reports to the file.

  // Increment this number if the format changes.
  DUMP(writer, "{\n  \"version\": 1,\n");

  DUMP(writer, "  \"hasMozMallocUsableSize\": ");

  nsCOMPtr<nsIMemoryReporterManager> mgr =
    do_GetService("@mozilla.org/memory-reporter-manager;1");
  NS_ENSURE_STATE(mgr);

  DUMP(writer, mgr->GetHasMozMallocUsableSize() ? "true" : "false");
  DUMP(writer, ",\n");
  DUMP(writer, "  \"reports\": ");

  // Process single reporters.
  bool isFirst = true;
  bool more;
  nsCOMPtr<nsISimpleEnumerator> e;
  mgr->EnumerateReporters(getter_AddRefs(e));
  while (NS_SUCCEEDED(e->HasMoreElements(&more)) && more) {
    nsCOMPtr<nsIMemoryReporter> r;
    e->GetNext(getter_AddRefs(r));

    nsCString process;
    rv = r->GetProcess(process);
    NS_ENSURE_SUCCESS(rv, rv);

    nsCString path;
    rv = r->GetPath(path);
    NS_ENSURE_SUCCESS(rv, rv);

    int32_t kind;
    rv = r->GetKind(&kind);
    NS_ENSURE_SUCCESS(rv, rv);

    int32_t units;
    rv = r->GetUnits(&units);
    NS_ENSURE_SUCCESS(rv, rv);

    int64_t amount;
    rv = r->GetAmount(&amount);
    NS_ENSURE_SUCCESS(rv, rv);

    nsCString description;
    rv = r->GetDescription(description);
    NS_ENSURE_SUCCESS(rv, rv);

    rv = DumpReport(writer, isFirst, process, path, kind, units, amount,
                    description);
    NS_ENSURE_SUCCESS(rv, rv);

    isFirst = false;
  }

  // Process multi-reporters.
  nsCOMPtr<nsISimpleEnumerator> e2;
  mgr->EnumerateMultiReporters(getter_AddRefs(e2));
  nsRefPtr<DumpMultiReporterCallback> cb = new DumpMultiReporterCallback();
  while (NS_SUCCEEDED(e2->HasMoreElements(&more)) && more) {
    nsCOMPtr<nsIMemoryMultiReporter> r;
    e2->GetNext(getter_AddRefs(r));
    r->CollectReports(cb, writer);
  }

  DUMP(writer, "\n  ]\n}");

  rv = writer->Finish();
  NS_ENSURE_SUCCESS(rv, rv);

#ifdef MOZ_DMD
  // Open a new file named something like
  //
  //   dmd-<identifier>-<pid>.txt.gz
  //
  // in NS_OS_TEMP_DIR for writing, and dump DMD output to it.  This must occur
  // after the memory reporters have been run (above), but before the
  // memory-reports file has been renamed (so scripts can detect the DMD file,
  // if present).

  nsCString dmdFilename;
  MakeFilename("dmd", aIdentifier, "txt.gz", dmdFilename);

  nsCOMPtr<nsIFile> dmdFile;
  rv = OpenTempFile(dmdFilename, getter_AddRefs(dmdFile));
  NS_ENSURE_SUCCESS(rv, rv);

  nsRefPtr<nsGZFileWriter> dmdWriter = new nsGZFileWriter();
  rv = dmdWriter->Init(dmdFile);
  NS_ENSURE_SUCCESS(rv, rv);

  // Dump DMD output to the file.

  DMDWriteState state(dmdWriter);
  dmd::Writer w(DMDWrite, &state);
  mozilla::dmd::Dump(w);

  rv = dmdWriter->Finish();
  NS_ENSURE_SUCCESS(rv, rv);
#endif  // MOZ_DMD

  // Rename the file, now that we're done dumping the report.  The file's
  // ultimate destination is "memory-report<-identifier>-<pid>.json.gz".

  nsCOMPtr<nsIFile> mrFinalFile;
  rv = NS_GetSpecialDirectory(NS_OS_TEMP_DIR, getter_AddRefs(mrFinalFile));
  NS_ENSURE_SUCCESS(rv, rv);

#ifdef ANDROID
  rv = mrFinalFile->AppendNative(NS_LITERAL_CSTRING("memory-reports"));
  NS_ENSURE_SUCCESS(rv, rv);
#endif

  rv = mrFinalFile->AppendNative(mrFilename);
  NS_ENSURE_SUCCESS(rv, rv);

  rv = mrFinalFile->CreateUnique(nsIFile::NORMAL_FILE_TYPE, 0600);
  NS_ENSURE_SUCCESS(rv, rv);

  nsAutoString mrActualFinalFilename;
  rv = mrFinalFile->GetLeafName(mrActualFinalFilename);
  NS_ENSURE_SUCCESS(rv, rv);

  rv = mrTmpFile->MoveTo(/* directory */ nullptr, mrActualFinalFilename);
  NS_ENSURE_SUCCESS(rv, rv);

  nsCOMPtr<nsIConsoleService> cs =
    do_GetService(NS_CONSOLESERVICE_CONTRACTID, &rv);
  NS_ENSURE_SUCCESS(rv, rv);

  nsString path;
  mrTmpFile->GetPath(path);
  NS_ENSURE_SUCCESS(rv, rv);

  nsString msg = NS_LITERAL_STRING(
    "nsIMemoryReporterManager::dumpReports() dumped reports to ");
  msg.Append(path);
  return cs->LogStringMessage(msg.get());
}

#undef DUMP
} // namespace mozilla


text:
from scdown.neo import (Neo,
                        NODE_USER,
                        NODE_TRACK,
                        NODE_COMMENT,
                        NODE_PROFILE,
                        REL_FOLLOWS,
                        REL_UPLOADED,
                        REL_FAVORITED,
                        REL_HAS_PROFILE,
                        REL_WROTE,
                        REL_REFERS_TO)
from py2neo import Graph
from itertools import product
from nose.tools import with_setup

TEST_DB = "http://127.0.0.1:8585/db/data/"
graph = Graph(TEST_DB)
neo = Neo(graph)

def setup_func():
    pass

def teardown_func():
    graph.delete_all()

datum = {"id": 1, "name": "Me"}
datum2 = dict(datum)
nested = {"new": {"data": True, "deeply": "nested"}}
datum2["novum"] = nested

def test_deflate():
    flat = neo.deflate(datum2)
    # adds keys due to nesting
    assert len(flat) == len(datum2) + 1
    # idempotent
    assert flat == neo.deflate(flat)

@with_setup(setup_func, teardown_func)
def test_create_or_update_node():
    datum = {"id": 1, "name": "Me"}
    datum1 = dict(datum)
    datum1["color"] = "red"
    node = neo.create_or_update_node(NODE_USER, datum)
    assert node.exists
    assert NODE_USER in node.labels
    node2 = neo.create_or_update_node(NODE_USER, datum1)
    assert node.ref == node2.ref
    assert node.properties == datum1

@with_setup(setup_func, teardown_func)
def test_node_types():
    nodes = set()
    for n in [NODE_USER, NODE_COMMENT, NODE_PROFILE, NODE_TRACK]:
        node = neo.create_or_update_node(n, datum)
        nodes.add(node)
    assert len(nodes) == 4

@with_setup(setup_func, teardown_func)
def test_relation_types():
    nodes = {}
    acceptable = set(
        [(NODE_USER, REL_HAS_PROFILE, NODE_PROFILE),
         (NODE_USER, REL_FOLLOWS, NODE_USER),
         (NODE_USER, REL_UPLOADED, NODE_TRACK),
         (NODE_USER, REL_FAVORITED, NODE_TRACK),
         (NODE_USER, REL_WROTE, NODE_COMMENT),
         (NODE_COMMENT, REL_REFERS_TO, NODE_TRACK)])
    accepted = set()
    rel_types = [REL_FOLLOWS,
                 REL_UPLOADED,
                 REL_FAVORITED,
                 REL_HAS_PROFILE,
                 REL_WROTE,
                 REL_REFERS_TO]
    for n in [NODE_USER, NODE_COMMENT, NODE_PROFILE, NODE_TRACK]:
        node = neo.create_or_update_node(n, datum)
        nodes[n] = node
    combos = product(nodes.items(), repeat=2)
    for c1, c2 in (tuple(prod) for prod in combos):
        k1, n1 = c1
        k2, n2 = c2
        for r in rel_types:
            try:
                neo.mk_relation(n1, r, n2)
                accepted.add((k1, r, k2))
            except AssertionError:
                pass
    assert acceptable == accepted

@with_setup(setup_func, teardown_func)
def test_nested_properties():
    node = neo.create_or_update_node(NODE_COMMENT, datum2)
    assert node.exists
    assert "novum" in node.properties
    assert node.properties["__json_novum"]
    assert neo.inflate(node.properties) == datum2


text:
#include <iostream>
#include <iomanip>
#include <fstream>
#include <assert.h>
#include <cstdlib>
#include <string>
#include <vector>
#include <queue>
#include <boost/regex.hpp>
#include <boost/tokenizer.hpp>
#include <boost/filesystem.hpp>
#include <boost/foreach.hpp>
#include <boost/algorithm/string.hpp>
#include "commands.h"
#include "Point.h"
#include "Vector.h"
#include "ProjPoint.h"
#include "TransformationWithFrames.h"
#include "TexShape.h"

using namespace std;
using namespace boost;

const double	 pi = 3.1415926535897;
const char polypathstr[] = "poly";
const char polyextstr[] = ".poly";
const char texpathstr[] = "tex";
boost::filesystem::path polypath(polypathstr);
boost::filesystem::path polyext(polyextstr);

Command::~Command() { }

token Command::tokenize_cmd(string command) {
	char_separator<char> seq(" \\");
	return token(command, seq);
}

void Command::writePolygons(polygonList polygons, string filename) {
	boost::filesystem::create_directory(polypathstr);
	ofstream outfile((string(polypathstr) + "/" + filename).c_str());
	//cout << (string(polypathstr) + "/" + filename).c_str() << " created\n";
	BOOST_FOREACH(vector<Point> polygon, polygons) {
		if (polygon.size() < 1)
			continue;
		BOOST_FOREACH(Point p, polygon) {
			outfile << p << " ";
		}
		outfile << endl;
	}
	outfile.close();
}

/**
 * store the command name and filename argument
 */
token::iterator preparse(token tok, Command &c) {
	token::iterator itr = tok.begin();
	c.cname = *itr; // command name
	assert(++itr != tok.end());

	// check for file extension
	cmatch what;
	regex exp("(.*)(\\.poly)");
	if (regex_match(((string)*itr).c_str(), what, exp)) { // file extension found
		c.filename = what[0];
	} else { // append file extension
		c.filename = ((string)*itr).append(polyextstr);
	}
	return ++itr;
}

/**
 * convert a transform code in string to enum type
 */
code_t codeStrToEnum(string code) {
	int index = -1;
	do {
		if (string(codeStr[++index]) == code)
			break;
	} while ((code_t)index != END);
	assert((code_t)index != END);
	code_t enumcode = (code_t)index;
	return enumcode;
}

MidasPolygon::MidasPolygon(token tok) {
	token::iterator itr = preparse(tok, *this);
	// insert the 2d points
	for (; itr != tok.end(); ++itr) {
		double x = atof( ((string)*itr).c_str() );
		assert(++itr != tok.end());
		double y = atof( ((string)*itr).c_str() );
		int r = 255, g = 255, b = 255;//, a = 255;
		token::iterator backtrackItr = itr;
		if (++itr != tok.end()) {
			string color = (string)*itr;
			if (color.size() == 6) {
				string red = string("0x").append(color.substr(0,2));
				string green = string("0x").append(color.substr(2,2));
				string blue = string("0x").append(color.substr(4,2));
				//string alpha = string("0x").append(color.substr(6,2));
				//cout << red << " " << green << " " << blue << endl;
				char *endptr;
				r = strtol(red.c_str(), &endptr, 16);
				g = strtol(green.c_str(), &endptr, 16);
				b = strtol(blue.c_str(), &endptr, 16);
				//a = strtol(alpha.c_str(), &endptr, 16);
			} else
				itr = backtrackItr;
		} else
			itr = backtrackItr;
		//cout << r << " " << g << " " << b << endl;
		Point p(x, y, 0.0, r, g, b);
		//cout << p << "\n";
		points.push_back(p);
		//cout << points.back() << "\n\n";
		assert(itr != tok.end());
	}
	//cout << endl;
}

void MidasPolygon::execute() {
	//cout << "Executing Polygon\n";
	boost::filesystem::create_directory(polypathstr);
	ofstream file((string(polypathstr) + "/" + this->filename).c_str());
	BOOST_FOREACH(Point p, this->points) {
		file << p << " ";
	}
	file << "\n";
	file.close();
}

Assemble::Assemble(token tok) {
	token::iterator itr = preparse(tok, *this);

	for (; itr != tok.end(); ++itr) {
		boost::filesystem::path filepath = polypath / (*itr + polyextstr);
		assert(boost::filesystem::exists(filepath));
		this->files.push_back(*itr);
	}
}

void Assemble::execute() {
	//cout << "Executing Assemble\n";
	boost::filesystem::create_directory(polypathstr);
	ofstream newfile((string(polypathstr) + "/" + this->filename).c_str());
	BOOST_FOREACH(string thisfile, files) {
		ifstream curfile((polypath / (thisfile + polyextstr)).string().c_str());
		char c;
		while ((c = curfile.get()) != EOF)
			newfile << c;
		curfile.close();
	}
	newfile.close();
	//cout << (string(polypathstr) + "/" + this->filename).c_str() << " assembled\n";
}

Transform::Transform() {

}

Transform::Transform(polygonList polygons) {
	this->polygons = polygons;
}

Transform::Transform(projPolygonList projPolygons) {
	this->projPolygons = projPolygons;
}

/**
 * transform constructor with argument
 * param tok tokenized command
 */
Transform::Transform(token tok) {
	token::iterator itr = preparse(tok, *this);
	readPolygons(this->filename, &polygons, &texturedPolygons);
	readCodes(tok, itr);
}

void Transform::readCodes(token tok, token::iterator itr) {
	for(; itr != tok.end(); itr++) {
		code_t code = codeStrToEnum(*itr);
		itr++;
		assert(itr != tok.end());
		tfCmd[code] = new double(atof((*itr).c_str()));
	}
}

vector<Point> getCoords(token tok) {
	vector<Point> newPolygon;
	vector<string> coords; // put all coords string in a vector
	BOOST_FOREACH(string s, tok) {
		//cout << s << " ";
		coords.push_back(s);
		////cout << coords.back() << " ";
	}
	//cout << endl;

	assert(coords.size() % 4 == 0);
	for (unsigned int i = 0; i < coords.size(); i += 4) {
		double x = atof(coords[i].c_str());
		double y = atof(coords[i+1].c_str());
		double z = atof(coords[i+2].c_str());
		string color = coords[i+3];
		assert(color.size() == 6);
		string red = string("0x").append(color.substr(0,2));
		string green = string("0x").append(color.substr(2,2));
		string blue = string("0x").append(color.substr(4,2));
		//string alpha = string("0x").append(color.substr(6,2));
		//cout << red << " " << green << " " << blue << endl;
		char *endptr;
		int r = strtol(red.c_str(), &endptr, 16);
		int g = strtol(green.c_str(), &endptr, 16);
		int b = strtol(blue.c_str(), &endptr, 16);
		//int a = strtol(alpha.c_str(), &endptr, 16);
		newPolygon.push_back(Point(x, y, z, r, g, b));
	}
	return newPolygon;
}

void Command::readPolygons(string filename, polygonList* polygons, texturedPolygonList* texturedPolygons) {
	//cout << "Reading " << filename << ":\n";
	char c[512];
	assert(boost::filesystem::exists(polypath / filename));
	//cout << (string(polypathstr) + "/" + filename).c_str() << endl;
	ifstream inf((string(polypathstr) + "/" + filename).c_str());
	while(inf.good()) {
		inf.getline(c,512);
		if (c[0] == '#' || c[0] == '\n') // comment or blank lines
			continue;
		if (c[0] == '\0')
			break;

		string str(c);
		erase_all(str, ")"); // remove extra characters
		erase_all(str, "(");
		replace_all(str, ",", " ");
//		cout << str << endl;

		char_separator<char> sep(" "); // tokenize line
		tokenizer<char_separator<char> > tok(str, sep);

		token::iterator itr = tok.begin();
		if (*itr == string("t")) {	// if textured polygon
			TexShape newTexShape;
			newTexShape.texturefile = *(++itr);
			int count = atoi((++itr)->c_str());
			//cout << count << endl;
			for (int i = 0; i < count; i++) {
				if (c[0] == '#' || c[0] == '\n') // comment or blank lines
					continue;
				if (c[0] == '\0')
					break;
				inf.getline(c,512);
				str = string(c);
				erase_all(str, ")"); // remove extra characters
				erase_all(str, "(");
				replace_all(str, ",", " ");
				tok = token(str, sep);
				newTexShape.shape.push_back(getCoords(tok));

				inf.getline(c,512);
				str = string(c);
				erase_all(str, ")"); // remove extra characters
				erase_all(str, "(");
				replace_all(str, ",", " ");
				tok = token(str, sep);
				newTexShape.texture.push_back(getCoords(tok));
			}
			//newTexShape.print(cout);
			if (texturedPolygons) {
				texturedPolygons->push_back(newTexShape);
			}
			continue;
		}

		vector<Point> newPolygon = getCoords(tok);
		if (newPolygon.size() > 0) // make sure no empty polygon is inserted
			polygons->push_back(newPolygon);
	}
	BOOST_FOREACH(vector<Point> polygon, *polygons) {
		BOOST_FOREACH(Point p, polygon) {
			//cout << p << " ";
		}
		//cout << endl;
	}
	inf.close();
}

void Transform::execute() {
	//cout << "Executing Transform\n";
	transform(translate() * scale() * rotate(), false);
}

polygonList Transform::transform(Transformation t, bool project) {
	//cout << "Transformation matrix:\n";
	//cout << t << endl;
	if (!project) { // transforming only
		vector< vector<Point> >::iterator polygonItr = polygons.begin();
		for(; polygonItr != polygons.end(); polygonItr++) {
			vector<Point>::iterator pointItr = (*polygonItr).begin();
			for(; pointItr != (*polygonItr).end(); pointItr++) // apply matrix to each points
				(*polygonItr).insert((*polygonItr).erase(pointItr),  t.applied_to(*pointItr));
		}
		for (int i = 0; i < texturedPolygons.size(); i++) {
			TexShape *ts = &texturedPolygons[i];
			vector< vector<Point> >::iterator polygonItr = ts->shape.begin();
			for(; polygonItr != ts->shape.end(); polygonItr++) {
				vector<Point>::iterator pointItr = (*polygonItr).begin();
				for(; pointItr != (*polygonItr).end(); pointItr++) // apply matrix to each points
					(*polygonItr).insert((*polygonItr).erase(pointItr),  t.applied_to(*pointItr));
			}
		}
		//cout << "polygons: \n";
//		BOOST_FOREACH(vector<Point> polygon, polygons) {
//			BOOST_FOREACH(Point p, polygon) {
//				cout << p << " ";
//			}
//			cout << endl;
//		}
		//cout << "textured polygons:\n";
//		BOOST_FOREACH(TexShape ts, texturedPolygons) {
//			BOOST_FOREACH(vector<Point> shape, ts.shape) {
//				BOOST_FOREACH(Point p, shape) {
//					cout << p << " ";
//				}
//				cout << endl;
//			}
//		}
	} else { // camera / viewing transform
		vector< vector<ProjPoint> >::iterator polygonItr = projPolygons.begin();
		for(; polygonItr != projPolygons.end(); polygonItr++) {
			vector<ProjPoint>::iterator pointItr = (*polygonItr).begin();
			for(; pointItr != (*polygonItr).end(); pointItr++)
				(*polygonItr).insert((*polygonItr).erase(pointItr),  t.applied_to(*pointItr));
		}

//		BOOST_FOREACH(vector<ProjPoint> polygon, projPolygons) {
//			BOOST_FOREACH(ProjPoint p, polygon) {
//				//cout << p << " ";
//			}
//			//cout << endl;
//		}

		polygons.clear();
		BOOST_FOREACH(vector<ProjPoint> polygon, projPolygons) {
			vector<Point> newpolygon;
			BOOST_FOREACH(ProjPoint p, polygon) {
//				if (p.w() < 0)
//					p.setW(-1.0 * p.w());
				//cout << p << " ";
				Point pp = p.project();
				newpolygon.push_back(pp);
			}
			polygons.push_back(newpolygon);
			//cout << endl;
		}
	}
	return this->polygons;
}

Transformation Transform::translate() {
	double x = (tfCmd[XT])?*tfCmd[XT]:0.000;
	double y = (tfCmd[YT])?*tfCmd[YT]:0.000;
	double z = (tfCmd[ZT])?*tfCmd[ZT]:0.000;
	//cout << codeStr[XT] << ": " << x << endl;
	//cout << codeStr[YT] << ": " << y << endl;
	//cout << codeStr[ZT] << ": " << z << endl;
	return Transformation::translate_by(x, y, z);
}

Transformation Transform::scale() {
	double x = (tfCmd[XS])?*tfCmd[XS]:1.000;
	double y = (tfCmd[YS])?*tfCmd[YS]:1.000;
	double z = (tfCmd[ZS])?*tfCmd[ZS]:1.000;
	double u = (tfCmd[US])?*tfCmd[US]:1.000;
	//cout << codeStr[XS] << ": " << x << endl;
	//cout << codeStr[YS] << ": " << y << endl;
	//cout << codeStr[ZS] << ": " << z << endl;
	//cout << codeStr[US] << ": " << u << endl;
	return Transformation::scale_by(x, y, z) * Transformation::scale_by(u, u, u);
}

Transformation Transform::rotate() {
	double xd = (tfCmd[XD])?*tfCmd[XD]:0.000;
	double yd = (tfCmd[YD])?*tfCmd[YD]:0.000;
	double zd = (tfCmd[ZD])?*tfCmd[ZD]:0.000;
	double xr = (tfCmd[XR])?*tfCmd[XR]:0.000;
	double yr = (tfCmd[YR])?*tfCmd[YR]:0.000;
	double zr = (tfCmd[ZR])?*tfCmd[ZR]:0.000;
	//cout << codeStr[XD] << ": " << xd << endl;
	//cout << codeStr[YD] << ": " << yd << endl;
	//cout << codeStr[ZD] << ": " << zd << endl;
	//cout << codeStr[XR] << ": " << xr << endl;
	//cout << codeStr[YR] << ": " << yr << endl;
	//cout << codeStr[ZR] << ": " << zr << endl;
	return Transformation::x_rotate_by(xd) * Transformation::y_rotate_by(yd) *
		Transformation::z_rotate_by(zd) * Transformation::x_rotate_by(xr * 180.0 / pi) *
		Transformation::y_rotate_by(yr * 180.0 / pi) * Transformation::z_rotate_by(zr * 180.0 / pi);
}

/**
 * create newfile oldfile transforms
 */
Create::Create(token tok) {
	token::iterator itr = preparse(tok, *this);
	outputfile = filename;

	assert(itr != tok.end());
	// check for file extension
	cmatch what;
	regex exp("(.*)(\\.poly)");
	if (regex_match(((string)*itr).c_str(), what, exp)) { // file extension found
		filename = what[0];
	} else { // append file extension
		filename = ((string)*itr).append(polyextstr);
	}
	readPolygons(filename, &polygons, &texturedPolygons);
	readCodes(tok, ++itr);
}

void Create::execute() {
	//cout << "Executing Create\n";
	Transform::execute();
	writePolygons(polygons, this->outputfile);
	BOOST_FOREACH(TexShape ts, texturedPolygons) {
		Surface::writeSurface(ts, outputfile);
	}
}

View::View() {

}

View::View(token tok) {
	token::iterator itr = preparse(tok, *this);
	eyeX = atof((*itr).c_str());
	assert(++itr != tok.end());
	eyeY = atof((*itr).c_str());
	assert(++itr != tok.end());
	eyeZ = atof((*itr).c_str());
	assert(++itr != tok.end());
	upX = atof((*itr).c_str());
	assert(++itr != tok.end());
	upY = atof((*itr).c_str());
	assert(++itr != tok.end());
	upZ = atof((*itr).c_str());
	assert(++itr != tok.end());
	centerX = atof((*itr).c_str());
	assert(++itr != tok.end());
	centerY = atof((*itr).c_str());
	assert(++itr != tok.end());
	centerZ = atof((*itr).c_str());
	assert(++itr != tok.end());
	angle = atof((*itr).c_str());
	assert(++itr != tok.end());
	near = atof((*itr).c_str());
	assert(++itr != tok.end());
	far = atof((*itr).c_str());
	readPolygons(filename, &polygons, &texturedPolygons);
}

void View::convert() {
	//cout << "Converting points to projective space\n";
	BOOST_FOREACH(vector<Point> polygon, polygons) {
		vector<ProjPoint> newpolygon;
		BOOST_FOREACH(Point p, polygon) {
			ProjPoint pp(p);
			newpolygon.push_back(pp);
			//cout << pp << " ";
		}
		projPolygons.push_back(newpolygon);
		//cout << endl;
	}
}

/**
 * view file eyex eyey eyez roll upy pitch centerx centery centerz angle near far
 */
void View::execute() {
//	convert();
	//cout << "Executing View\n";
//	Frame f1(Vector(1, 0, 0), Vector(0, 1, 0), Vector(0, 0, 1), Point(centerX, centerY, centerZ));
//	Vector w = Point(eyeX, eyeY, eyeZ) - Point(centerX, centerY, centerZ);
//	w.normalize();
//	Frame f2(w, Vector(upX,1,0), Point(eyeX, eyeY, eyeZ));
//	Transform tf(projPolygons);
//	Transformation camera_t = Transformation::frame_to_frame(f1, f2);
//	Transformation viewing_t = Transformation::view(near, far, angle);
//	//cout << "Camera:\n" << camera_t << endl;
//	//cout << "Viewing:\n" << viewing_t << endl;
//	polygons = tf.transform(camera_t * viewing_t, true);
	//cout << "Final Points:\n";
//	BOOST_FOREACH(vector<Point> polygon, polygons) {
//		BOOST_FOREACH(Point p, polygon) {
//			//cout << p << " ";
//		}
//		//cout << endl;
//	}
}

Extrude::Extrude(token tok) {
	token::iterator itr = preparse(tok, *this);
	assert(itr != tok.end());

	cmatch what;
	regex exp("(.*)(\\.poly)");
	if (regex_match(((string)*itr).c_str(), what, exp)) { // file extension found
		newfile = what[0];
	} else { // append file extension
		newfile = ((string)*itr).append(polyextstr);
	}
	assert(++itr != tok.end());
	units = atof((*itr).c_str());
	readPolygons(filename, &polygons);
}

/**
 * extrude oldfile newfile units
 */

void Extrude::execute() {
	//cout << "Executing Extrude\n";
	vector<vector<Point> > newPolygons;

	BOOST_FOREACH(vector<Point> polygon, polygons) {
		newPolygons.push_back(polygon); // add the original base
		vector<Point> topFace;
		Vector normal;
		for (unsigned int i = 0; i < polygon.size(); i++) {
			if (i == 0) { // get the normal vector from first 3 points
				Vector v1;
				Vector v2;
				v1 = polygon.back() - polygon[0];
				//cout << "v1: " << v1 << endl;
				v2 = polygon[i+1] - polygon[i];
				v1.normalize();
				v2.normalize();
				//cout << "v2: " << v2 << endl;
				normal = cross(v1, v2);
				normal.normalize();
				//cout << "normal: " << normal << endl;
				normal *= units;
			}
			topFace.push_back(polygon[i] + normal); // add the extruded points
		}
//		BOOST_FOREACH(Point p, topFace) {
//			//cout << p << " ";
//		}
		//cout << endl;
		newPolygons.push_back(topFace); // add extruded face

		// add the extruded sides
		for (unsigned int i = 0; i < polygon.size(); i++) {
			vector<Point> newSideFace;
			newSideFace.push_back(polygon[i]); // push 2 base points
			if (i+1 < polygon.size())
				newSideFace.push_back(polygon[i+1]);
			else
				newSideFace.push_back(polygon[0]);
			if (i+1 < topFace.size()) // push 2 top face points
				newSideFace.push_back(topFace[i+1]);
			else
				newSideFace.push_back(topFace[0]);
			newSideFace.push_back(topFace[i]);

			newPolygons.push_back(newSideFace); // add extruded sides
		}

//		BOOST_FOREACH(vector<Point> polygon, newPolygons) {
//			BOOST_FOREACH(Point p, polygon) {
//				//cout << p << " ";
//			}
//			//cout << endl;
//		}

		writePolygons(newPolygons, this->newfile);

	}
}

Clip::Clip(token tok) {
	token::iterator itr = preparse(tok, *this);
	assert(itr != tok.end());

	cmatch what;
	regex exp("(.*)(\\.poly)");
	if (regex_match(((string)*itr).c_str(), what, exp)) { // file extension found
		newfile = what[0];
	} else { // append file extension
		newfile = ((string)*itr).append(polyextstr);
	}
	double px, py, pz, nx, ny, nz;
	assert(++itr != tok.end());
	px = atof((*itr).c_str());
	assert(++itr != tok.end());
	py = atof((*itr).c_str());
	assert(++itr != tok.end());
	pz = atof((*itr).c_str());
	assert(++itr != tok.end());
	nx = atof((*itr).c_str());
	assert(++itr != tok.end());
	ny = atof((*itr).c_str());
	assert(++itr != tok.end());
	nz = atof((*itr).c_str());
	p = Point(px, py, pz);
	vnorm = Vector(nx, ny, nz);
	vnorm.normalize();

	readPolygons(filename, &polygons);
}

void Clip::execute() {
	//cout << "Executing Clip" << endl;
	// get the clipping point
	//cout << "p: " << p << endl << "vnorm: " << vnorm << endl;
	polygonList clippedPolygons;
	BOOST_FOREACH(vector<Point> polygon, polygons) {
		vector<Point> clippedPolygon;
		Point lastPoint = polygon.back();
		BOOST_FOREACH(Point q, polygon) {
			float d1 = dot(vnorm, (q-p));
			float d2 = dot(vnorm, (lastPoint - p));
			//cout << "d1: " << d1 << " d2: " << d2 << endl;
			if (d1 >= 0.0 && d2 >= 0.0) {
				clippedPolygon.push_back(q);
				//cout << q << " is IN" << endl;
			}
			else if (d1 >= 0.0 && d2 < 0.0 || d1 < 0.0 && d2 >= 0.0) { // lastpoint on outside
				float t = d1 / (d1 - d2);
				Point I = q + t*(lastPoint - q);
				Point *outPoint; // determine which one is out to set color
				if (d1 >= 0.0) { // current is IN, last is OUT
					//cout << q << " is IN\n";
					outPoint = &lastPoint;
				} else { // current is OUT, last is IN
					//cout << q << " is OUT\n";
					outPoint = &q;
				}
				I.setA(outPoint->a()); // set color of intersection point
				I.setG(outPoint->g()); //  to the color of point on
				I.setR(outPoint->r()); //  outside
				I.setB(outPoint->b());
				//cout << I << " is an intersection\n";
				clippedPolygon.push_back(I); // add intersection point to polygon
				if (d1 >= 0.0) // add current point to polygon if IN
					clippedPolygon.push_back(q);
			} else if (d1 == 0.0) { // point is on the plane
				clippedPolygon.push_back(q);
				//cout << q << " is on the plane\n";
			} else {
				//cout << q << " is OUT\n";
			}
			lastPoint = q;
		}
		clippedPolygons.push_back(clippedPolygon);
	}

	//cout << "clipped polygons:\n";
//	BOOST_FOREACH(vector<Point> clippedPolygon, clippedPolygons) {
//		BOOST_FOREACH(Point cp, clippedPolygon) {
//			//cout << cp << " ";
//		}
//		//cout << endl;
//	}

	writePolygons(clippedPolygons, this->newfile);
}

Volume::Volume(token tok) {
	token::iterator itr = preparse(tok, *this);
	assert(itr != tok.end());

	cmatch what;
	regex exp("(.*)(\\.poly)");
	if (regex_match(((string)*itr).c_str(), what, exp)) { // file extension found
		newfile = what[0];
	} else { // append file extension
		newfile = ((string)*itr).append(polyextstr);
	}
	assert(++itr != tok.end());
	double px = atof((*itr).c_str());
	assert(++itr != tok.end());
	double py = atof((*itr).c_str());
	assert(++itr != tok.end());
	double pz = atof((*itr).c_str());
	assert(++itr != tok.end());
	double vx = atof((*itr).c_str());
	assert(++itr != tok.end());
	double vy = atof((*itr).c_str());
	assert(++itr != tok.end());
	double vz = atof((*itr).c_str());
	assert(++itr != tok.end());
	alpha = atof((*itr).c_str());
	assert(++itr != tok.end());
	resolution = atof((*itr).c_str());

	referencePoint = Point(px, py, pz);
	axis = Vector(vx, vy, vz);

	readPolygons(filename, &polygons);
}

void Volume::execute() {
	//cout << "Executing Volume\n";

	polygonList newPolygons;

	//cout << "\tTranslating to origin...\n";
	Point p1 = -referencePoint;
	Transformation translateTransform = Transformation::translate_by(p1.x(), p1.y(), p1.z());
	Transform* transformCmd = new Transform(polygons);
	polygons = transformCmd->transform(translateTransform, false);

	Transformation stepRotate = Transformation::rotate_by(resolution, axis);
	//cout << "\t" << stepRotate << endl;

	polygonList lastPolygons = polygons;
	double curRotated = 0.0;

	for (; curRotated <= alpha; curRotated += resolution) {
		//cout << "\tRotated by " << curRotated << " degrees\n";
		polygons = transformCmd->transform(stepRotate, false);
		for(unsigned int i = 0; i < polygons.size(); i++) {
			vector<Point> polygon = polygons[i];
			vector<Point> lastpolygon = lastPolygons[i];
			newPolygons.push_back(lastpolygon);
			if (curRotated + resolution >= alpha)
				newPolygons.push_back(polygon);

			for(unsigned int j = 0; j < polygon.size(); j++) {
				vector<Point> newSideFace;
				newSideFace.push_back(polygon[j]); // push 2 base points
				if (j+1 < polygon.size())
					newSideFace.push_back(polygon[j+1]);
				else
					newSideFace.push_back(polygon[0]);
				if (j+1 < lastpolygon.size()) // push 2 top face points
					newSideFace.push_back(lastpolygon[j+1]);
				else
					newSideFace.push_back(lastpolygon[0]);
				newSideFace.push_back(lastpolygon[j]);

				newPolygons.push_back(newSideFace); // add surfaces
			}

		}
		lastPolygons = polygons;
	}
	if (curRotated < alpha) {
		// generate one more volume to close surface
	}

	//cout << "\tTranslating back to original position...\n";
	Point p0 = referencePoint;
	translateTransform = Transformation::translate_by(p0.x(), p0.y(), p0.z());
	if (transformCmd) delete transformCmd;
	transformCmd = new Transform(newPolygons);
	newPolygons = transformCmd->transform(translateTransform, false);

	writePolygons(newPolygons, newfile);
}

Surface::Surface(token tok) {
	token::iterator itr = preparse(tok, *this);
	assert(itr != tok.end());
	texturefile = *itr;
	itr++;
	int level = atoi((*itr).c_str());
	assert(level >= 0);

	matrix<Point> patch(4,4);

	for (int i = 0; i < 4; i++) {
		for (int j = 0; j < 4; j++) {
			assert(++itr != tok.end());
			int x = atoi((*itr).c_str());
			assert(++itr != tok.end());
			int y = atoi((*itr).c_str());
			assert(++itr != tok.end());
			int z = atoi((*itr).c_str());
			patch(i,j) = Point(x,y,z);
		}
	}
	surface = new BezierPatch(patch);
	surface->refine(level);
}

void Surface::execute() {
	cout << "Executing Surface..." << endl;
	matrix<Point> patch = surface->flatten();
	for (unsigned int i = 0; i < patch.RowNo()-1; i++) {
		for (unsigned int j = 0; j < patch.ColNo()-1; j++) {
			vector<Point> polygon;
			vector<Point> texCoords;
			polygon.push_back(Point(patch(i,j))); // top left
			texCoords.push_back( Point(
					(double)j/((double)patch.ColNo()-1.0),
					1.0-(double)i/((double)patch.RowNo()-1.0) ) );
			polygon.push_back(Point(patch(i,j+1))); // top right
			texCoords.push_back( Point(
					(double)(j+1)/((double)patch.ColNo()-1.0),
					1.0-(double)i/((double)patch.RowNo()-1.0) ) );
			polygon.push_back(Point(patch(i+1,j+1))); // bottom right
			texCoords.push_back( Point(
					(double)(j+1)/((double)patch.ColNo()-1.0),
					1.0-(double)(i+1)/((double)patch.RowNo()-1.0) ) );
			polygon.push_back(Point(patch(i+1,j))); // bottom left
			texCoords.push_back( Point(
					(double)j/((double)patch.ColNo()-1.0),
					1.0-(double)(i+1)/((double)patch.RowNo()-1.0) ) );
			polygons.shape.push_back(polygon);
			polygons.texture.push_back(texCoords);
			polygons.texturefile = texturefile;
		}
	}
	writeSurface(polygons, filename, ios_base::trunc);
}

void Surface::writeSurface(TexShape polygons, string filename, ios_base::openmode mode) {
	boost::filesystem::create_directory(polypathstr);
	ofstream outfile((string(polypathstr) + "/" + filename).c_str(), mode);
	polygons.print(outfile);
	outfile.close();
	cout << (string(polypathstr) + "/" + filename).c_str() << " created\n";
}