text:
<?xml version="1.0" encoding="UTF-8"?>

<!--
  ~ Licensed to the Apache Software Foundation (ASF) under one
  ~ or more contributor license agreements. See the NOTICE file
  ~ distributed with this work for additional information
  ~ regarding copyright ownership. The ASF licenses this file
  ~ to you under the Apache License, Version 2.0 (the
  ~ "License"); you may not use this file except in compliance
  ~ with the License. You may obtain a copy of the License at
  ~
  ~ http://www.apache.org/licenses/LICENSE-2.0
  ~
  ~ Unless required by applicable law or agreed to in writing,
  ~ software distributed under the License is distributed on an
  ~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
  ~ KIND, either express or implied. See the License for the
  ~ specific language governing permissions and limitations
  ~ under the License.
  -->

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">

    <parent>
        <groupId>org.apache.rampart</groupId>
        <artifactId>rampart-project</artifactId>
        <version>1.8.0-SNAPSHOT</version>
        <relativePath>../../pom.xml</relativePath>
    </parent>

    <modelVersion>4.0.0</modelVersion>
    <artifactId>rampart-integration</artifactId>
    <packaging>jar</packaging>
    <name>Rampart - Integration</name>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-dependency-plugin</artifactId>
                <executions>
                    <execution>
                        <id>copy-endorsed</id>
                        <phase>process-resources</phase>
                        <goals>
                            <goal>copy-dependencies</goal>
                        </goals>
                        <configuration>
                            <includeGroupIds>org.apache.xerces,org.apache.xalan</includeGroupIds>
                            <outputDirectory>${project.build.directory}/endorsed</outputDirectory>
                            <stripVersion>true</stripVersion>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.axis2</groupId>
                <artifactId>axis2-repo-maven-plugin</artifactId>
                <executions>
                    <execution>
                        <id>create-test-repo</id>
                        <goals>
                            <goal>create-test-repository</goal>
                        </goals>
                        <configuration>
                            <modules>addressing,rampart</modules>
                            <outputDirectory>target/test-classes</outputDirectory>
                        </configuration>
                    </execution>
                    <execution>
                        <id>create-rampart-client-repo</id>
                        <goals>
                            <goal>create-test-repository</goal>
                        </goals>
                        <configuration>
                            <modules>addressing,rampart,rahas</modules>
                            <outputDirectory>target/test-resources/rampart_client_repo</outputDirectory>
                        </configuration>
                    </execution>
                    <execution>
                        <id>create-rampart-service-repo</id>
                        <goals>
                            <goal>create-test-repository</goal>
                        </goals>
                        <configuration>
                            <modules>addressing,rampart,rahas</modules>
                            <axis2xml>src/test/resources/conf/axis2.xml</axis2xml>
                            <outputDirectory>target/test-resources/rampart_service_repo</outputDirectory>
                        </configuration>
                    </execution>
                    <execution>
                        <id>create-rahas-client-repo</id>
                        <goals>
                            <goal>create-test-repository</goal>
                        </goals>
                        <configuration>
                            <modules>addressing,rampart</modules>
                            <outputDirectory>target/test-resources/rahas_client_repo</outputDirectory>
                        </configuration>
                    </execution>
                    <execution>
                        <id>create-rahas-service-repo-1</id>
                        <goals>
                            <goal>create-test-repository</goal>
                        </goals>
                        <configuration>
                            <modules>addressing,rampart,rahas</modules>
                            <axis2xml>src/test/resources/conf/axis2.xml</axis2xml>
                            <outputDirectory>target/test-resources/rahas_service_repo_1</outputDirectory>
                        </configuration>
                    </execution>
                    <execution>
                        <id>create-rahas-service-repo-3</id>
                        <goals>
                            <goal>create-test-repository</goal>
                        </goals>
                        <configuration>
                            <modules>addressing,rampart,rahas</modules>
                            <axis2xml>src/test/resources/conf/axis2.xml</axis2xml>
                            <outputDirectory>target/test-resources/rahas_service_repo_3</outputDirectory>
                        </configuration>
                    </execution>
                    <execution>
                        <id>create-default-security-client-repo</id>
                        <goals>
                            <goal>create-test-repository</goal>
                        </goals>
                        <configuration>
                            <modules>addressing,rampart</modules>
                            <axis2xml>src/test/resources/conf/axis2.xml</axis2xml>
                            <configurationDirectory>conf</configurationDirectory>
                            <outputDirectory>target/test-resources/default_security_client_repo</outputDirectory>
                        </configuration>
                    </execution>
                    <execution>
                        <id>create-rahas-service-repo-5</id>
                        <goals>
                            <goal>create-test-repository</goal>
                        </goals>
                        <configuration>
                            <modules>addressing,rampart,rahas</modules>
                            <axis2xml>src/test/resources/conf/axis2.xml</axis2xml>
                            <outputDirectory>target/test-resources/rahas_service_repo_5</outputDirectory>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-antrun-plugin</artifactId>
                <executions>

                    <execution>
                        <id>create-test-resources</id>
                        <phase>process-test-resources</phase>
                        <configuration>
                            <tasks>
                                <property name="addressing.mar" value="addressing-${axis2.version}.mar" />

                                <mkdir dir="target/temp-ramp" />
                                <mkdir dir="target/temp-ramp/META-INF" />

                                <copy overwrite="yes" file="src/test/resources/rampart/store.jks" tofile="target/temp-ramp/store.jks" />
                                <!--path id="ramp.client.props" location="test-resources/rampart"/-->
                                <!--maven:addPath id="maven.dependency.classpath" refid="ramp.client.props" -->
                                <mkdir dir="target/test-resources/rampart_client_repo" />
                                <mkdir dir="target/test-resources/rampart_client_repo/conf" />
                                <mkdir dir="target/test-resources/rampart_service_repo" />
                                <mkdir dir="target/test-resources/rampart_service_repo/conf" />
                                <mkdir dir="target/test-resources/rampart_service_repo/services" />

                                <!-- Service 1 -->
                                <copy overwrite="yes" file="src/test/resources/rampart/services-1.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureService1.aar" basedir="target/temp-ramp" />
                                <!-- Service 2 -->
                                <copy overwrite="yes" file="src/test/resources/rampart/services-2.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureService2.aar" basedir="target/temp-ramp" />
                                <!-- Service 3 -->
                                <copy overwrite="yes" file="src/test/resources/rampart/services-3.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureService3.aar" basedir="target/temp-ramp" />
                                <!-- Service 4 -->
                                <copy overwrite="yes" file="src/test/resources/rampart/services-4.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureService4.aar" basedir="target/temp-ramp" />
                                <!-- Service 5 -->
                                <copy overwrite="yes" file="src/test/resources/rampart/services-5.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureService5.aar" basedir="target/temp-ramp" />
                                <!-- Service 6 -->
                                <copy overwrite="yes" file="src/test/resources/rampart/services-6.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureService6.aar" basedir="target/temp-ramp" />
                                <!-- Service 7 -->
                                <copy overwrite="yes" file="src/test/resources/rampart/services-7.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureService7.aar" basedir="target/temp-ramp" />
                                <!-- Service 8 -->
                                <copy overwrite="yes" file="src/test/resources/rampart/services-8.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureService8.aar" basedir="target/temp-ramp" />
                                <!-- Service 9 -->
                                <copy overwrite="yes" file="src/test/resources/rampart/services-9.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureService9.aar" basedir="target/temp-ramp" />
                                <!-- Service 10 -->
                                <copy overwrite="yes" file="src/test/resources/rampart/services-10.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureService10.aar" basedir="target/temp-ramp" />
                                <!-- Service 11 -->
                                <copy overwrite="yes" file="src/test/resources/rampart/services-11.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureService11.aar" basedir="target/temp-ramp" />
                                <!-- Service 12 -->
                                <copy overwrite="yes" file="src/test/resources/rampart/services-12.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureService12.aar" basedir="target/temp-ramp" />
                                <!-- Service 13 -->
                                <copy overwrite="yes" file="src/test/resources/rampart/services-13.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureService13.aar" basedir="target/temp-ramp" />
                                <!-- Service 14 -->
                                <copy overwrite="yes" file="src/test/resources/rampart/services-14.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureService14.aar" basedir="target/temp-ramp" />
                                <!-- Service 15 -->
                                <copy overwrite="yes" file="src/test/resources/rampart/services-15.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureService15.aar" basedir="target/temp-ramp" />
                                <!-- Service 16 -->
                                <copy overwrite="yes" file="src/test/resources/rampart/services-16.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureService16.aar" basedir="target/temp-ramp" />
                                <!-- Service 17 -->
                                <copy overwrite="yes" file="src/test/resources/rampart/services-17.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureService17.aar" basedir="target/temp-ramp" />
                                <!-- Service 18 -->
                                <copy overwrite="yes" file="src/test/resources/rampart/services-18.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureService18.aar" basedir="target/temp-ramp" />
                                <!-- Service 19 -->
                                <copy overwrite="yes" file="src/test/resources/rampart/services-19.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureService19.aar" basedir="target/temp-ramp" />
                                <!-- Service 20 -->
                                <copy overwrite="yes" file="src/test/resources/rampart/services-20.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureService20.aar" basedir="target/temp-ramp" />
                                <!-- Service 21 -->
                                <copy overwrite="yes" file="src/test/resources/rampart/services-21.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureService21.aar" basedir="target/temp-ramp" />
                                <!-- Service 22 -->
                                <copy overwrite="yes" file="src/test/resources/rampart/services-22.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureService22.aar" basedir="target/temp-ramp" />
                                <!-- Service 23 -->
                                <copy overwrite="yes" file="src/test/resources/rampart/services-23.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureService23.aar" basedir="target/temp-ramp" />
                                <!-- Service 24 -->
                                <copy overwrite="yes" file="src/test/resources/rampart/services-24.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureService24.aar" basedir="target/temp-ramp" />
                               <!-- Service 25 -->
                                <copy overwrite="yes" file="src/test/resources/rampart/services-25.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureService25.aar" basedir="target/temp-ramp" />
                                <!-- Service 26 -->
                                <copy overwrite="yes" file="src/test/resources/rampart/services-26.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureService26.aar" basedir="target/temp-ramp" />
                                 <!-- Service 27 -->
                                <copy overwrite="yes" file="src/test/resources/rampart/services-27.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureService27.aar" basedir="target/temp-ramp" />
                                <!-- Service 28 -->
                                <copy overwrite="yes" file="src/test/resources/rampart/services-28.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureService28.aar" basedir="target/temp-ramp" />
                                <!-- Service 29 -->
                                <copy overwrite="yes" file="src/test/resources/rampart/services-29.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureService29.aar" basedir="target/temp-ramp" />
                                <!-- Service 30 -->
                                <copy overwrite="yes" file="src/test/resources/rampart/services-30.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureService30.aar" basedir="target/temp-ramp" />

                                <!-- Service 31 -->
                                <copy overwrite="yes" file="src/test/resources/rampart/services-31.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureService31.aar" basedir="target/temp-ramp" />

                                <!-- Service 32 -->
                                <copy overwrite="yes" file="src/test/resources/rampart/services-32.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureService32.aar" basedir="target/temp-ramp" />

                                <!-- Service 33 -->
                                <copy overwrite="yes" file="src/test/resources/rampart/services-33.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureService33.aar" basedir="target/temp-ramp" />

                                <!-- Service 34 -->
                                <copy overwrite="yes" file="src/test/resources/rampart/services-34.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureService34.aar" basedir="target/temp-ramp" />

                                <!-- Service 35 -->
                                <copy overwrite="yes" file="src/test/resources/rampart/services-35.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureService35.aar" basedir="target/temp-ramp" />

                                <!-- Service SC-1 -->
                                <copy overwrite="yes" file="src/test/resources/rampart/issuer.properties" tofile="target/temp-ramp/issuer.properties" />
                                <copy overwrite="yes" file="src/test/resources/rampart/services-sc-1.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureServiceSC1.aar" basedir="target/temp-ramp" />

                                <!-- Service SC-2 -->
                                <copy overwrite="yes" file="src/test/resources/rampart/issuer.properties" tofile="target/temp-ramp/issuer.properties" />
                                <copy overwrite="yes" file="src/test/resources/rampart/services-sc-2.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureServiceSC2.aar" basedir="target/temp-ramp" />

                                <!-- Service SC-3 -->
                                <copy overwrite="yes" file="src/test/resources/rampart/issuer.properties" tofile="target/temp-ramp/issuer.properties" />
                                <copy overwrite="yes" file="src/test/resources/rampart/services-sc-3.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureServiceSC3.aar" basedir="target/temp-ramp" />

                                <!-- Service SC-4 This is with standard secure conversation specification -->
                                <copy overwrite="yes" file="src/test/resources/rampart/issuer.properties" tofile="target/temp-ramp/issuer.properties" />
                                <copy overwrite="yes" file="src/test/resources/rampart/services-sc-4.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureServiceSC4.aar" basedir="target/temp-ramp" />

                                <!-- Service SC-5 This is with standard secure conversation specification -->
                                <copy overwrite="yes" file="src/test/resources/rampart/issuer.properties" tofile="target/temp-ramp/issuer.properties" />
                                <copy overwrite="yes" file="src/test/resources/rampart/services-sc-5.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureServiceSC5.aar" basedir="target/temp-ramp" />

                                <!-- Service SC-6 This is with standard secure conversation specification -->
                                <copy overwrite="yes" file="src/test/resources/rampart/issuer.properties" tofile="target/temp-ramp/issuer.properties" />
                                <copy overwrite="yes" file="src/test/resources/rampart/services-sc-6.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/SecureServiceSC6.aar" basedir="target/temp-ramp" />

                                <!--  Kerberos Services -->
                                <copy overwrite="yes" file="src/test/resources/rampart/kerberos/KerberosOverTransportKeytab.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/KerberosOverTransportKeytab.aar" basedir="target/temp-ramp" />

                                <copy overwrite="yes" file="src/test/resources/rampart/kerberos/KerberosOverTransportPWCB.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/KerberosOverTransportPWCB.aar" basedir="target/temp-ramp" />

                                <copy overwrite="yes" file="src/test/resources/rampart/kerberos/KerberosDelegation.xml" tofile="target/temp-ramp/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rampart_service_repo/services/KerberosDelegation.aar" basedir="target/temp-ramp" />

                                <!--
                                 Set up the infra for rahas tests and the rahas client repo
                                -->
                                <mkdir dir="target/temp-rahas" />
                                <mkdir dir="target/temp-rahas/META-INF" />
                                <copy overwrite="yes" todir="target/temp-rahas">
                                    <fileset dir="src/test/resources/rahas">
                                        <include name="issuer.properties" />
                                        <include name="rahas-sts.jks" />
                                    </fileset>
                                </copy>
                                <!--path id="rahas.client.props" location="test-resources/rahas" -->
                                <!--addPath id="maven.dependency.classpath" refid="rahas.client.props"-->
                                <mkdir dir="target/test-resources/rahas_client_repo" />
                                <mkdir dir="target/test-resources/rahas_client_repo/conf" />
                                <!-- Rahas Test1: SAML Token test -->
                                <mkdir dir="target/test-resources/rahas_service_repo_1" />
                                <mkdir dir="target/test-resources/rahas_service_repo_1/conf" />
                                <mkdir dir="target/test-resources/rahas_service_repo_1/services" />
                                <!-- copy the services.xml and create the aar -->
                                <copy overwrite="yes" file="src/test/resources/rahas/s1-services.xml" tofile="target/temp-rahas/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rahas_service_repo_1/services/SecureService.aar" basedir="target/temp-rahas" />
                                <!-- Rahas Test1: SAML Token test : END -->
                                <!--
                                 Rahas Test 3 & 4: RahasSAMLTokenUTForHoKTest and RahasSAMLTokenUTForHoKV1205Test
                                -->
                                <mkdir dir="target/test-resources/rahas_service_repo_3" />
                                <mkdir dir="target/test-resources/rahas_service_repo_3/conf" />
                                <mkdir dir="target/test-resources/rahas_service_repo_3/services" />
                                <!-- copy the services.xml and create the aar -->
                                <copy overwrite="yes" file="src/test/resources/rahas/s3-services.xml" tofile="target/temp-rahas/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rahas_service_repo_3/services/SecureService.aar" basedir="target/temp-rahas" />
                                <!--
                                 Rahas Test 3 & 4: RahasSAMLTokenUTForHoKTest and RahasSAMLTokenUTForHoKV1205Test : END
                                -->
                                <!--
                        RahasSAMLTokenAttributeTest
                        -->
                                <mkdir dir="target/test-resources/rahas_service_repo_5" />
                                <mkdir dir="target/test-resources/rahas_service_repo_5/conf" />
                                <mkdir dir="target/test-resources/rahas_service_repo_5/services" />
                                <!-- copy the services.xml and create the aar -->
                                <copy overwrite="yes" file="src/test/resources/rahas/s5-services.xml" tofile="target/temp-rahas/META-INF/services.xml" />
                                <jar jarfile="target/test-resources/rahas_service_repo_5/services/SecureService.aar" basedir="target/temp-rahas" />

                                <!--RahasAttributeTest END-->
                            </tasks>
                        </configuration>
                        <goals>
                            <goal>run</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <!--plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>build-helper-maven-plugin</artifactId>
                <executions>
                    <execution>
                        <id>add-test-source</id>
                        <phase>process-resources</phase>
                        <goals>
                            <goal>add-test-source</goal>
                        </goals>
                        <configuration>
                            <sources>
				<source>target/generated-code/resources</source>
			    </sources>
                        </configuration>
                    </execution>
                </executions>
            </plugin-->
            <plugin>
                <groupId>com.github.veithen.alta</groupId>
                <artifactId>alta-maven-plugin</artifactId>
                <executions>
                    <execution>
                        <goals>
                            <goal>generate-properties</goal>
                        </goals>
                        <configuration>
                            <name>aspectjweaver</name>
                            <value>%file%</value>
                            <artifacts>
                                <artifact>
                                    <groupId>org.aspectj</groupId>
                                    <artifactId>aspectjweaver</artifactId>
                                </artifact>
                            </artifacts>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <artifactId>maven-surefire-plugin</artifactId>
                <configuration>
                    <!-- Add the Xerces/Xalan versions expected by OpenSAML to the boot classpath so that the build succeeds on older 1.5 JDKs -->
                    <argLine>${jacoco.surefireArgLine} -Xbootclasspath/p:${project.build.directory}/endorsed/xml-apis.jar${path.separator}${project.build.directory}/endorsed/xercesImpl.jar${path.separator}${project.build.directory}/endorsed/resolver.jar${path.separator}${project.build.directory}/endorsed/serializer.jar${path.separator}${project.build.directory}/endorsed/xalan.jar -javaagent:${aspectjweaver}</argLine>
                </configuration>
            </plugin>
            <plugin>
                <artifactId>maven-deploy-plugin</artifactId>
                <configuration>
                    <skip>true</skip>
                </configuration>
            </plugin>
        </plugins>
    </build>

    <dependencies>
        <dependency>
            <groupId>org.apache.rampart</groupId>
            <artifactId>rampart-policy</artifactId>
            <version>${project.version}</version>
            <!--  Use Jetty's servlet api (2.5) -->
            <exclusions>
                <exclusion>
                    <groupId>javax.servlet</groupId>
                    <artifactId>servlet-api</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>org.apache.rampart</groupId>
            <artifactId>rampart-trust</artifactId>
            <version>${project.version}</version>
            <!--  Use Jetty's servlet api (2.5) -->
            <exclusions>
                <exclusion>
                    <groupId>javax.servlet</groupId>
                    <artifactId>servlet-api</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>org.apache.rampart</groupId>
            <artifactId>rampart-core</artifactId>
            <version>${project.version}</version>
            <!--  Use Jetty's servlet api (2.5) -->
            <exclusions>
                <exclusion>
                    <groupId>javax.servlet</groupId>
                    <artifactId>servlet-api</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>org.apache.rampart</groupId>
            <artifactId>rampart-testutils</artifactId>
            <version>${project.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.axis2</groupId>
            <artifactId>axis2-transport-http</artifactId>
            <version>${axis2.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.axis2</groupId>
            <artifactId>axis2-transport-local</artifactId>
            <version>${axis2.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.axis2</groupId>
            <artifactId>axis2-adb</artifactId>
            <version>${axis2.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.rampart</groupId>
            <artifactId>rampart</artifactId>
            <version>${project.version}</version>
            <type>mar</type>
        </dependency>
        <dependency>
            <groupId>org.apache.rampart</groupId>
            <artifactId>rahas</artifactId>
            <version>${project.version}</version>
            <type>mar</type>
        </dependency>
        <dependency>
            <groupId>org.apache.axis2</groupId>
            <artifactId>addressing</artifactId>
            <type>mar</type>
        </dependency>
        <dependency>
            <groupId>org.apache.directory.server</groupId>
            <artifactId>apacheds-core-annotations</artifactId>
            <version>2.0.0-M21</version>
            <scope>test</scope>
            <exclusions>
                <exclusion>
                    <groupId>bouncycastle</groupId>
                    <artifactId>bcprov-jdk15</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>org.apache.directory.server</groupId>
            <artifactId>apacheds-protocol-shared</artifactId>
            <version>2.0.0-M21</version>
            <scope>test</scope>
            <exclusions>
                <exclusion>
                    <groupId>bouncycastle</groupId>
                    <artifactId>bcprov-jdk15</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>org.apache.directory.server</groupId>
            <artifactId>apacheds-protocol-kerberos</artifactId>
            <version>2.0.0-M21</version>
            <scope>test</scope>
            <exclusions>
                <exclusion>
                    <groupId>bouncycastle</groupId>
                    <artifactId>bcprov-jdk15</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>org.apache.directory.server</groupId>
            <artifactId>apacheds-interceptor-kerberos</artifactId>
            <version>2.0.0-M21</version>
            <scope>test</scope>
            <exclusions>
                <exclusion>
                    <groupId>bouncycastle</groupId>
                    <artifactId>bcprov-jdk15</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>org.apache.directory.api</groupId>
            <artifactId>api-ldap-codec-standalone</artifactId>
            <version>1.0.0-M33</version>
        </dependency>
        <dependency>
            <groupId>org.apache.directory.api</groupId>
            <artifactId>api-ldap-extras-codec-api</artifactId>
            <version>1.0.0-M33</version>
        </dependency>
        <dependency>
            <groupId>commons-io</groupId>
            <artifactId>commons-io</artifactId>
            <version>2.4</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>commons-collections</groupId>
            <artifactId>commons-collections</artifactId>
            <version>3.2</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.ws.commons.axiom</groupId>
            <artifactId>axiom-truth</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-log4j12</artifactId>
        </dependency>
        <dependency>
            <groupId>log4j</groupId>
            <artifactId>log4j</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.axis2</groupId>
            <artifactId>axis2-testutils</artifactId>
            <version>${axis2.version}</version>
        </dependency>
    </dependencies>
</project>


text:
Home/Sports/"I came here to play the best tennis, but I didn't think about it." – Tennis Sports
"I came here to play the best tennis, but I didn't think about it." – Tennis Sports
christinacodyDecember 2, 2021
Twenty major champion Rafael Nadal led the field at the last Masters 1000 event in Paris last year. Following the 13th crown at the French Open, Nadal was uncertain about the competition between the Paris Masters and the ATP Finals.
Still, there is no world. 2 decided to attend the Masters 1000 event in the French capital in search of one of the rare and notable titles lacking in his extensive collection. Rafa made his debut in Paris in 2007 and reached the final round, but only won four games against David Nalbandian to win the runner-up prize.
Nadal, who had been injured in the months following the US Open, had never been popular on indoor hardcourts and made three more semi-finals in Paris, but again competed for the title. There wasn't. In 2019, Rafa went on to the final four after defeating Jo-Wilfried Tsonga and withdrew due to an abdominal injury prior to Denis Shapovalov's clash, but still scored valuable points at the end of the year. increase.
One battle with Novak Djokovic. After a proper and natural rest, Rafa won consecutive crowns in the French capital last November, following the success of the historic Roland Garros, with defending champion Novak Djokovic skipping Paris. I stood as a top seed when I decided.
Nadal played in five tournaments in 2020, before Paris, and won two ATP crowns at Acapulco and Roland Garros. Nadal spent six months at home, left the tennis courts in March and April, began practicing in May, and did not like the situation in New York ahead of the Cincinnati and US Open.
Rafael Nadal was the top seed of the Rolex Masters in Paris last year.
The Spaniards skipped both events, maintaining 2000 points in the 2019 US Open title and looking to Rome and Roland Garros. Rafa was unable to lift another trophy at Foroi Tarico and was defeated by Diego Schwartzman in the quarter, entering an unknown territory of Paris, dealing with cold and rainy weather and the roof above Philippe-Chatrie.
Standing on top of all obstacles and rivals, Nadal claimed the crown of the 13th Roland Garros, never lost the set and joined Roger Federer with 20 major trophies. In search of the 36th Masters 1000 Crown, Rafa reached the semi-finals of Paris before losing to Alexander Zverev on the straight set.
After dealing with a leg injury, the Spaniards had to skip this year's edition of the last Masters 1000 event of the season. "I'm here to do my best not to get injured. When I go to tournaments, I concentrate on playing my best tennis.
Don't worry if I've been injured in the past, it can happen again. We are certainly in a different situation than the previous year. Still, you need to be aware of the situation and be prepared to accept the inconvenience. If nothing happens in Paris, I'll be in the ATP Finals.
It's part of the history of our game. This is an important event and is the best gathering of players of the season. I like playing tennis, so I like to play more than one tournament every 6 or 7 months. So it's not new to play well without playing tennis after a while. I did it in the past. It confirms that I can do it, "Rafael Nadal said.
– Tennis Sports
"I came here to play the best tennis, but I didn't think about it."
https://www.tennisworldusa.org/tennis/news/Rafael_Nadal/105807/rafael-nadal-recalls-i-came-here-to-play-my-best-tennis-and-not-think-about-/ "I came here to play the best tennis, but I didn't think about it."
christinacody
AEW Dynamite Summary and Reaction (December 1, 2021): Cody's on fire - WWE Sports
Jon Jones resigns from former coach and calls Mike Winkel John a "miserable bastard" - MMA Sports
"Dismissing Steve Bruce at this critical stage is a big mistake and almost certainly demotes." – Soccer Sports
Titans secures top seed, AFC's only first round buy – NFL Sports
Giants revive John Brebbia and move Aaron Sanchez to 60-day IL – MLB Sports
Harish English, personally the best in Tennessee – Tennis Sports
Alonzo Menifield hopes KO will beat Ed Herman in the UFC 265. – MMA Sports
Boston Red Sox pitcher Chris Sale positive for COVID-19 test – MLB Sports

redpajama_set_name: RedPajamaCommonCrawl


Binary Options News and promotions.
BinaryOptionsCanada.net provides a complete binary options trading guide for Canadian investors. This guide includes news & reviews on binary options brokers serving Canada, taxes, education & strategies.


text:
\chapter{Introduction}
\title{MantissaCam: Learning Snapshot High-dynamic-range Imaging with Perceptually-based In-pixel Irradiance Encoding}

\author{Haley M. So$^1$ $\quad$ Julien N.P. Martel$^1$ $\quad$ Piotr Dudek$^2$ $\quad$ Gordon Wetzstein$^1$\\
$^1$Stanford University $\quad$ $^2$The University of Manchester\\
{\tt\small \href{https://www.computationalimaging.org/publications/mantissacam/}{computationalimaging.org/publications/mantissacam/} }
}
\date{}
\maketitle

\begin{abstract}
The ability to image high-dynamic-range (HDR) scenes is crucial in many computer vision applications. The dynamic range of conventional sensors, however, is fundamentally limited by their well capacity, resulting in saturation of bright scene parts. To overcome this limitation, emerging sensors offer in-pixel processing capabilities to encode the incident irradiance. Among the most promising encoding schemes is modulo wrapping, which results in a computational photography problem where the HDR scene is computed by an irradiance unwrapping algorithm from the wrapped low-dynamic-range (LDR) sensor image. Here, we design a neural network--based algorithm that outperforms previous irradiance unwrapping methods and, more importantly, we design a perceptually inspired ``mantissa'' encoding scheme that more efficiently wraps an HDR scene into an LDR sensor. Combined with our reconstruction framework, MantissaCam achieves state-of-the-art results among modulo-type snapshot HDR imaging approaches. We demonstrate the efficacy of our method in simulation and show preliminary results of a prototype MantissaCam implemented with a programmable sensor.
\end{abstract}

\section{Introduction}
\label{sec:intro}
\input{latex/sections/introduction}

\section{Related Work}
\label{sec:related}
\input{latex/sections/related}

\section{Perceptually-based HDR Imaging}
\label{sec:method}
\input{latex/sections/method}

 \section{Experiments}
\label{sec:experiments}
\input{latex/sections/experiments}

\section{Discussion}
\label{sec:discussion}
\input{latex/sections/discussion}

\section*{Acknowledgements}
This project was in part supported by NSF Award 1839974, the NSF Graduate Research Fellowship, and a PECASE by the ARL.

\section*{\Large Supplemental Material\\MantissaCam: Learning Snapshot High-dynamic-range Imaging with Perceptually-based In-pixel Irradiance Encoding}
\end{center}

\begin{multicols}{2}
\renewcommand{\thefigure}{S\arabic{figure}}
\renewcommand{\thesection}{S\arabic{section}}
\renewcommand{\thetable}{S\arabic{table}}

\section{Pipeline Details}

\subsection{Mantissa Dataset creation}
There are several ways to encode the mantissa. When working with synthetic data, the simplest way is to just take the $\log$ of the signal and then take the modulo of the resulting value. Recall $\log$ of anything below 1 is a negative value, which would not be conceivable with the hardware. Instead, only after the pixel saturates do we take the $\log$ to simulate the mantissa. When we saturate the pixel, the subsequent wrap will require twice the intensity to wrap again. To create the training dataset, we create the mantissa image and the corresponding winding number image. For each pixel $ij$,
\begin{equation}
        \text{mantissa}_{ij}=
\begin{cases}
    I_{ij},& \text{if } I_{ij} < I_{max}\\
    \log_{\alpha}(I_{ij}) \%I_{max},& \text{otherwise.}
\end{cases}
\end{equation}

\begin{equation}
    \text{winding number}_{ij}=
\begin{cases}
    0,&\text{if } I_{ij} < I_{max}\\
    \big\lfloor \log_{\alpha}(I_{ij}) \big\rfloor + 1,& \text{otherwise.}
\end{cases}
\end{equation}
where $\%$ denotes the modulo operation and $\left\lfloor \cdot \right\rfloor$ denotes the floor function. For our dataset and experiments, we set  $I_{max} = 1$ and $\alpha = 2$.

\subsection{Network Architecture}
In this subsection, we describe the architecture for the single pass winding number prediction network (also see Figure~\ref{fig:AttentionNetwork}). The extracted edges from the edge prediction network, along with the mantissa image, are fed into the network via feature extraction by a $7 \times 7$ convolutional layer, an instance norm, relu, and a non-local block for the extracted edge features. These images are then concatenated and sent through a squeeze-and-excitation block to perform dynamic channel-wise feature recalibration. The base network is an attention unet, pioneered by Oktay et al.~\cite{oktay2018attention}. The backbone is the U-Net where the expanding path has attention gates added, along with the skip connections. Skip connections allow features extracted from the contracting path to be used in the expanding path. The attention block places more emphasis on the features of the skip connections.

\begin{figure*}[t]
\begin{center}
   {\includegraphics[ width=\linewidth]{latex/figures/AttentionNetwork.pdf}}
\end{center}
   \caption{Attention UNet architecture of the winding number prediction network. }
\label{fig:AttentionNetwork}
\end{figure*}

\section{Training Procedures}

\subsection{RGB training on HDR images}
For training our network for RGB, we trained the edge network for 400 epochs on the synthetic data at learning rate of .0001 using an ADAM optimizer in Pytorch. From a dataset of 593 images, we randomly split it into 400 training images and 193 test images. We augment the training images by scaling the HDR image and calculating the corresponding mantissa and winding numbers.

\subsection{Training Procedure for SCAMP-5 Prototype}
We retrain the edge prediction network for grayscale images for 200 epochs and the winding number prediction network for 200 epochs. For working with hardware with unknown noise, we added an additional downsampling and upsampling layer in the attention-unet. To match the noise of the SCAMP-5 more closely, we inject a gaussian noise prior to simulating the mantissa sensor image. More characterization of the SCAMP-5 is needed to improve on the reconstruction of the mantissa images, but preliminary results are promising, as shown in the extended Figure~\ref{fig:extendedScamp}.

\begin{figure*}[t]
\begin{center}
   \includegraphics[trim=0cm 15cm 0cm 0cm, width=1\linewidth]{latex/figures/extendedScamp.pdf}
\end{center}
   \caption{Extended SCAMP-5 captured results and reconstruction. The top row shows the directly captured images by the SCAMP-5 sensor. The bottom row shows the reconstructed Mantissa, the tonemapped HDR image reconstructed from bracketed exposures, the tonemapped HDR image reconstructed from the captured modulo and winding number, and the tonemapped HDR using the mantissa and the predicted winding number.   }
\label{fig:extendedScamp}
\end{figure*}

\subsection{Baseline Comparison}
\paragraph{Graph Cuts.}
Graph Cuts was implemented following the original ModuloCam paper~\cite{Zhao:2015} using the same custom potential function. Reaching out to the authors confirmed the method, which can be successful for some clearly wrapped modulo images, however requires delicate parameter tuning for each of the many layer unwrappings of each image. We chose a set of parameters to best unwrap the whole set. PSNR, SSIM, and MSSIM scores were comparable to those found in UnModNet~\cite{zhou2020unmodnet} when they implemented the MRF algorithm.
\paragraph{Modulo Encoding with UnModNet.}
We retrained UnModNet, the state of the art for unwrapping modulo images, with the same training process and same dataset as in \cite{zhou2020unmodnet} and results were comparable to those reported in the paper. In areas of dense wrappings, the pipeline struggles to stop unwrapping, leading to patches of white.
\paragraph{Mantissa Encoding with UnModNet.}
One of our baseline experiment is to use the pipeline of UnModNet with the forward imaging model of MantissaCam. We trained the pipeline using the same training procedure as described above. We noted the layer-by-layer unwinding did not work well with the reconstruction from the mantissa encoding as errors in winding number manifest in exponentially bad errors. Indeed, missing a wrap results in much worse errors in MantissaCam (because of the exponential function used when reconstructing) than in ModuloCam, resulting in huge artifacts. Besides, the nature of the layer-by-layer unwrapping is prone to propagating errors.
\paragraph{Modulo Encoding with Our Network.}
To combat propagation of errors in unwinding, we directly predict the winding number in a single pass through an attention-unet instead of predicting a mask. Again, we train using the same training procedure as UnModNet. Results are promising, however, the network still struggles when the modulo image has very tight wrappings (of the order of 1--2 pixels width).
\paragraph{Mantissa Encoding with Our Network.}
Introducing the mantissa allows us to spatially spread out the wraps as we get to higher and higher irradiance levels. This leads to preservation of more detail. Results comparing these methods, excluding the UnModNet for the mantissa, are shown in the paper and in the additional figures in Section~4.

\subsection{Additional Implementation Details}
We compare the full reconstructed HDR image with the ground truth HDR image to calculate PSNR and Q-Score (2). We then tonemap both the ground truth and the predicted HDR images, all using the Reinhard Algorithm with gamma = 1, intensity = 1. The tonemapped images are then compared to calculate SSIM and MSSIM values. Inference time for our method is much faster than for the UnModNet or graph cuts due to the single pass architecture, as opposed to the iterative unwrapping that can unwrap as high as the default of 15 max iterations.

\section{Additional Details of Experimental Results}
Currently, mantissa images cannot be directly captured in SCAMP-5. However we implemented a procedure on SCAMP-5 to capture modulo images and record the winding number. The modulo and winding numbers can then be recombined in a post-processing stage to obtain a full HDR image (using the reconstruction formula for the modulo image: $I = I_{\mathrm{sensor}} + W\cdotI_{\mathrm{max}}$). From the irradiance $I$ recovered this way, the mantissa and corresponding winding numbers can be recalculated, thus emulating MantissaCam with plausible noise. This is because we do not have a log-photosensor front-end in SCAMP-5 (even though it existed in a previous SCAMP-3 version) nor the ability to divide the gain of each pixel individually.

We also implemented a bracketed exposure procedure directly on the camera in order to get ground truth irradiance. Exposure bracketing is performed by capturing 5 images doubling the exposure time between each exposure, starting from a configurable short exposure time.

Figure~\ref{fig:extendedScamp} shows (from left to right, and top to bottom) both the lowest exposure and highest exposures captured with exposure bracketing, the modulo exposure as captured by SCAMP-5, the winding number also captured and recorded by SCAMP-5, the mantissa image computed from the modulo and winding numbers, the tone-mapped HDR image reconstructed from the exposure bracketing using Debevec algorithm \cite{Debevec:1997}, the tone-mapped HDR image reconstructed by adding the modulo and winding number, and the tone mapped reconstructed mantissa image using our neural network. All the tone-mapped images are computed using the Local Laplacian filter and tone-mapping described in \cite{paris2011local}.

\section{Additional Results}
See Figures~\ref{fig:supplement1} and ~\ref{fig:supplement2} for additional results. From left to right, each row shows the modulo image, the graph cuts method, UnModNet + modulo, Ours + modulo, the mantissa image, Ours + mantissa, and the ground truth image. All tonemapped images follow the tone-mapping described in Section~3.

\begin{figure*}[!h]
\begin{center}
   \includegraphics[width=0.95\linewidth]{latex/figures/supp1.pdf}
\end{center}
   \caption{More results showing the comparison between different baselines and encodings. Ours + mantissa is better able to keep details in the high intensity areas. PSNR (P), SSIM (S), and Q-Scores (Q) are shown about the reconstructed images.}
\label{fig:supplement1}
\end{figure*}

\begin{figure*}[!h]
\begin{center}
   \includegraphics[width=0.95\linewidth]{latex/figures/supp2.pdf}
\end{center}
   \caption{More results comparing the different reconstruction and encoding methods.}
\label{fig:supplement2}
\end{figure*}

\subsection{In-pixel Irradiance Encoding}

The image formation model of the MantissaCam is
\begin{equation}
	I_{\textrm{sensor}} \left(x,y \right) = q \left( \textrm{mod} \left( \textrm{log}_\alpha \left( I \left(x,y \right) \right), I_{\textrm{max}} \right) \right) + \eta,
\end{equation}
where $I$ describes the spatially varying irradiance (i.e., the target HDR image) on the sensor, $I_{\textrm{sensor}}$ is the measured LDR sensor image, and $\eta$ is zero-mean additive Gaussian noise. The parameter $\alpha$ models a family of logarithmic irradiance response functions. For example, the special case $\alpha=2$ of our encoding scheme is similar to the mantissa encoding of the IEEE 754 standard for floating point arithmetic. Sensor quantization is modeled by the function $q( \cdot )$. $I_{\textrm{max}}$ is the maximum allowed irradiance value before the intensity wraps. This could be the well capacity of a pixel or a user-defined value that is slightly lower than that.

\subsection{Irradiance Decoding}

The proposed decoding scheme is implemented by two neural networks. The first takes the wrapped sensor image as input and predicts the wrap edges, effectively separating them from the texture edges. The second network predicts the winding number (i.e., the number of times intensity has wrapped) of each pixel from these wrap edges.

To predict either modulo or mantissa wrap edges from a sensor image, we directly use the ``modulo edge separator'' proposed as part of the UnModNet architecture~\cite{zhou2020unmodnet}. This edge separator is a residual-type convolutional neural network (CNN) that takes as input a concatenation of the LDR sensor image and a Laplacian-filtered copy of the same. We illustrate our network in Figure~\ref{fig:pipeline} and refer the interested reader to~\cite{zhou2020unmodnet} for additional details.

Our second network predicts the winding number for each pixel, $W(x,y)$, given the wrap edges and the sensor image as input. For this purpose, features are extracted from both input images using the lightweight CNN-based feature extraction layers from~\cite{zhou2020unmodnet}. These are fed into an attention UNet~\cite{oktay2018attention} with four downsampling and four upsampling blocks, with each downsampling block using a strided convolutional layer and a residual bottleneck block, and each upsampling block mirroring it but with the addition of attention gates. This is a standard neural network architecture, but its application to directly predicting the winding number of irradiance-wrapped images is new. Note that this part of our algorithm is substantially different from the iterative, graph-cuts inspired unwrapping procedure proposed in~\cite{zhou2020unmodnet}. Their method aims at unwrapping the HDR image layer by layer, which is prone to propagating errors, whereas our approach directly predicts the number of wraps, i.e., the winding number, using a single pass through the UNet. We discuss additional details of this network architecture in the supplement and outline the training procedure of both networks in Section~\ref{sec:methods:details} and the supplement.

Given the predicted winding number for each pixel as well as the raw sensor $I_{\textrm{sensor}}$, we formulate the reconstruction of the HDR image $\widetilde{I}$ as
\begin{equation}
    \widetilde{I} \left(x,y \right) = \alpha^{ I_{\textrm{sensor}} + W(x,y) \cdot I_{\textrm{max}}}.
\end{equation}
In our implementation, we choose $\alpha = 2$.

\subsection{Understanding the Relation between Resolution and Dynamic Range}

The theory addressing the ability to perfectly reconstruct a signal with MantissaCam falls within the framework of unlimited sampling recently developed in \cite{bhandari2017unlimited,bhandari2020unlimited}. Here, rather than formally treating the reconstruction problem, we attempt to highlight the advantages of a mantissa over a modulo encoding and develop an understanding of the tradeoffs between those.
\newcommand{I_{\mathrm{max}}}{I_{\mathrm{max}}}
\newcommand{\mathrm{mod}}{\mathrm{mod}}
\newcommand{\mathrm{mant}}{\mathrm{mant}}
\newcommand{\mathcal{W}_\modu}{\mathcal{W}_\mathrm{mod}}
\newcommand{\mathcal{W}_\mant}{\mathcal{W}_\mathrm{mant}}
\newcommand{\mathcal{W}}{\mathcal{W}}
\newcommand{\mathrm{d}}{\mathrm{d}}

Let us consider the 1D band-limited irradiance function $I(x)$, with maximal frequency $f_{\mathrm{max}}$. The irradiance is encoded on the sensor by the wrapping function $\mathcal{W}$ of the imaging model:
\begin{equation}
    \mathcal{W}:I\in\mathbb{R}_+ \mapsto \mathcal{W}(I) \in [0,I_{\mathrm{max}}].
    \label{eq:modulo}
\end{equation}
In particular, we consider the two wrapping functions:
\begin{equation}
    \mathcal{W}_\modu(I) = I - W\big(I(x)\big)\cdot I_{\mathrm{max}},
    \label{eq:mantissa}
\end{equation}
and
\begin{equation}
  \mathcal{W}_\mant(I) = \log_\alpha(I) - W\big(I(x)\big)\cdot I_{\mathrm{max}},
\end{equation}
with $W\big(I(x)\big)=\left\lfloor \frac{I(x)}{I_{\mathrm{max}}} \right\rfloor = W\big(x\big)$ and $\lfloor \cdot \rfloor$ being the floor function.

In order to avoid aliasing on our discrete sensor array, we assume the sampling of $I$ respects the Nyquist sampling criterion $f_s>2\cdot f_{\mathrm{max}}$, with the sampling frequency $f_s$ related to the inverse pixel pitch $T_s=\frac{1}{f_s}$ (i.e. the resolution or pixel density, for instance expressed in line pairs per millimeter) of the sensor array.

\paragraph{Recoverability of irradiance from modulo and mantissa encodings}
To get an intuition about the irradiance fields $\mathcal{W}(I)$ that can be perfectly reconstructed, let us consider the discretized irradiance $I[n]=I(n\cdot T_s)$ as seen by a pixel $n$.

If a wrap of $\mathcal{W}(I)$ occurs within a pixel, information is lost and it is impossible to reconstruct the incident irradiance field. Therefore, a set of conditions to recover the field is:
\begin{equation}
\begin{cases}
    \Big| \mathcal{W}(I[n+1])-\mathcal{W}(I[n]) \Big| \leq I_{\mathrm{max}}, \\
    W(I[n+1])-W(I[n]) \leq 1,
\end{cases}
\end{equation}
where the first condition derives from the Euclidean Division Theorem and makes sure we cannot wrap "within" a pixel, the second condition allows at most one wrap between two pixels.

For the modulo encoding those conditions translate into
\begin{equation}
   \big| I[n+1] - I[n] \big| \leq I_{\mathrm{max}},
\end{equation}
and for the mantissa encoding we have that
\begin{equation}
   \big| \log_\alpha(I[n+1]) - \log_\alpha(I[n]) \big| \leq I_{\mathrm{max}}.
\end{equation}

This shows that while the modulo encoding can reconstruct any irradiance with arithmetic growth of $I_{\mathrm{max}}$, a mantissa encoding can reconstruct a larger class of functions with geometric growth of $I_{\mathrm{max}}$.

\paragraph{Dynamic range.} For both types of encoding, these results imply an interesting tradeoff between the dynamic range of the sensor and its spatial resolution. With two sensors of the same size, using different pixel pitches $T_s$ and $T_s'$ such that $T_s'>T_s$, the sensor with a smaller pixel pitch $T_s$ (i.e., of higher resolution) can reconstruct faster spatial variations of irradiance ($\frac{I_{\mathrm{max}}}{T_s}>\frac{I_{\mathrm{max}}}{T_s'}$ in the modulo case). Therefore, there is a relationship between the maximum dynamic range recoverable for a sensor given its resolution. For two sensors of fixed size with $N$ pixels, the maximum recoverable irradiance is a ramp starting at pixel $n=0$ and ending at pixel $n=N-1$. In this setting, the sensor with modulo encoding can reconstruct a maximum dynamic range of $\mathrm{DR}\approx 10\log(N\cdotI_{\mathrm{max}})$dB while the one with a mantissa encoding can recover a much wider dynamic range of $\mathrm{DR}\approx 10\cdot N\log(I_{\mathrm{max}})$dB.

\paragraph{Quantization.}
The ultra-high dynamic range of the mantissa encoding comes at the expense of loss of precision. In practice, no sensor has infinite bit depth but is quantized to 8--12~bits. As shown in the bottom graphs of Figure~\ref{fig:histogram}, the same number of levels are distibuted on a much wider range as the winding number
$W$ increases. This means a MantissaCam cannot resolve irradiance with the same precision ModuloCam can at high irradiance levels---the quantization error is higher for our encoding. Yet, early psycho-physics studies~\cite{fechner1948elements} noted that perceived light intensity is proportional to the logarithm of the light intensity. Known as Fechner-Weber law, this implies that the coarser quantization of MantissaCam at high irradiance levels might not be perceptually important.

\subsection{Dataset and Implementation Details}
\label{sec:methods:details}

For a fair comparison, the dataset used to train and evaluate our model was the same dataset created by UnModNet~\cite{zhou2020unmodnet}. We randomly split the images into 400 training images and 193 testing images. We used the same process to augment the training dataset, over-exposing and cropping images to yield a total of 5,945 training images.

We train our networks in three stages. First, we train the wrap edge prediction network by itself for 400 epochs, taking simulated sensor images as input, using a binary cross entropy loss with the ground truth wrap edge images obtained via simulation. Second, we train the winding number prediction network by itself for 200 epochs, taking simulated sensor images and ground truth wrap edges as input, using a mean-squared error (MSE) loss on the ground truth winding number. Third, we train both networks end-to-end for another 200 epochs using the same MSE loss on ground truth winding number. Additional implementation details are found in the supplement.

\endinput

\subsection{The reconstruction pipeline}

To reconstruct the hdr image, we need to predict how many times the pixel well is reset. But, before predicting the winding number, we want to be able to distinguish the edges of the natural image and the edges due to windings. Using architecture described in section 3, we train our edge predictor network on images from the hdr dataset []. The mantissa image, with the help of a laplace kernel, is inputted and labeled with a binary image of edges corresponding to the edges due to wrapping. We train for []. The edge information with the mantissa image is then inputted into the second network to predict the winding numbers. We train for []. Afterwards, we train both end-to-end. Table 1 shows the training parameters and specifics of the pipeline. We implement the networks in pytorch, train for [] epochs etc.

\subsection{Mantissa Irradiance Encoding}

Human eyes perceive changes in light on a logarithmic scale, meaning, we are much more sensitive to slight shifts in brightness at lower intensities than at higher intensities. We can jointly design the hardware and software to utilize this key insight. So inspired by the human perception of light and the modulo camera, we empty and recollect the photoelectrons whenever we saturate the well. Unlike modulo cam, however, each time, it takes twice the amount to reset the well, mimicking the logarithmic scale.
The output of this camera is the modulo of the log2 of the intensity, also known as the mantissa in IEEE Floating Point Standard 754. Let $m$ be the mantissa, or what is captured by this camera. Pixel at $x,y$ can be simulated by
\begin{align}
    m_{x,y} = && &(\log_2{I}) \% 1 \quad \text{if $I>1$}\\

        && &\text{(well, it may be a bit diff depending}\\
        && &\text{on how we want to think about it)}
\end{align}
where $I$ is the full intensity.

In the images collected, we get the signature wraps seen in modulo images, but the effect of the logarithmic wrapping is the reduction of dense wrappings, allowing us to recover a larger class of images and with less ambiguity since the wraps are more spread out and distinct. In figure 1, we show the modulo and the mantissa of the image displaying this quality. In regions of extremely high intensity, the modulo camera exhibits tight and dense wrappings, almost looking like noise. In the mantissa camera, the wraps are more spread out and easier to distinguish. In theory, the modulo camera allows for unbounded dynamic range, however in practice, the dense wraps make it incredibly difficult to reconstruct. Reconstruction becomes the limiting factor in dynamic range. The mantissa, on the other hand, will allow for a much higher dynamic range due to the wrap spacing that results with this kind of light collection. We're able to successfully reconstruct a much larger class of images and at a higher accuracy.

\subsection{Network-based HDR Image Reconstruction}

\textcolor{red}{GW: include some implementation details here, including network architecture, loss function, training details, and info on dataset}

The goal is to now recover the full dynamic range from the wrapped image. Like in the modulo camera paper, the crucial part of reconstruction is two fold: predicting the winding number and the reconstruction itself. While modulo camera uses graph cuts, recent deep learning techniques show promising results in unwrapping phase. The main challenges to the graph-cuts algorithm is that there is ambiguity between edges that result from the image windings and the edges in the scene. There can also regions of rapidly oscillating values and when unwrapped channel by channel, there can be color inconsistencies. UnModNet, specifically designed to unwrap modulo images, combatted these challenges using deep networks with great results. We adapted UnModNet to reconstruct mantissa images. The pipeline features two networks: one to predict which edges are the wrapping edges, and one to predict the number of wraps each pixel undergoes.
\\\\\\
 ... go into more detail about the specific network architectures once we decide what we want .... and differentiate between UnModeNet....
\\\\\\\\
In the end, the hdr image can be reconstructed with the predicted wrap number through the inverse of equation 1. The recovered value, $\phi$, at a pixel is
\begin{equation}
    \phi = 2^{p +m}- 1
\end{equation}
where $p$ is the predicted number of wraps and $m$ is the measured value from the camera. \emph{(also might be different depending on how we want to think about eqn 1.)} Once the predicted winding number is outputted from the network, we can combine it with the measured value to unwrap the image.

How well the images could be unwrapped, whether using a modulo camera or a mantissa camera, depends on how closely the windings occurr. In signal processing, a wrapped signal can be unwrapped without ambiguity if the sample rate is high enough. However, when undersampled, we get aliasing. The same principal applies to images. Using the modulo camera, at high intensities, the wrappings occur at such high rates that if the intensity increased greater than what can be resolved by the pixel width, we will get aliasing. This can be seen in modulo images near the brightest regions where the intensity increases faster than the pixel width. The benefit of the mantissa camera is that the wraps get sparser and sparser at high intensities, so we do not run into the same issue that modulo camera experiences. This allows us to recover a larger class of images and makes the recovery easier and more robust to noise.

\subsection{Evaluation on Synthetic Data}

Figure~\ref{fig:results} qualitatively and quantitatively compares modulo and mantissa encoding schemes combined with different reconstruction algorithms. Using a single modulo-wrapped image as input, graph cuts perform poorly~\cite{Zhao:2015}. The UnModNet network~\cite{zhou2020unmodnet} does reasonably well in some cases, but struggles to reconstruct the large bright parts of the first example scene and the lights on the bridge of the third scene. {Their iterative unwrapping procedure sometimes fails in stopping to unwrap, which results in extremely high irradiance values lowering their PSNR and obscuring fine image detail.} Our algorithm achieves a better quality than these methods on the same modulo-encoded images, as evaluated by the peak signal-to-noise (PSNR or P), structural similarity (SSIM or S), and Q-score of the perceptual HDR-VDP-2~\cite{Mantiuk:2011} metrics. Moreover, when combined with the proposed mantissa irradiance encoding scheme, our framework achieves the best results among all of these methods.

Table~\ref{tab:quantitative} also quantitatively compares all of these approaches using many different metrics on the test set of the dataset described in Sec.~\ref{sec:methods:details}. In addition to the above methods, we also include a comparison to a CNN operating directly on a conventional LDR sensor image to hallucinate the HDR scene~\cite{HDRCNN}. Again, the proposed mantissa encoding scheme combined with our reconstruction framework achieves the best results using all metrics, outperforming the state of the art, i.e., UnModNet, by almost $8$~dB of PSNR.

\subsection{Prototype MantissaCam}

We build a physical prototype, using an example of an emerging class of sensors dubbed focal-plane sensor--processors~\cite{zarandy2011focal}. Those embed small processing circuits inside each pixel. We use SCAMP-5~\cite{carey2013100}, whose processing elements (PE) are programmable in a single instruction multiple data (SIMD) fashion, similar to a GPU where the same instruction is performed for all processing elements simultaneously on some local piece of data. Specifically, a PE is equipped with a few analog and digital memories. Instructions can be performed as light is being collected by the pixel's photo-sensitive element, thus enabling to change the way integration is performed, as required for our MantissaCam implementation.

In our implementation, an exposure consists of $N$ iterations of length $\delta{t}$ (that is programmable and sets the total exposure as $\Delta{T}=N\cdot{\delta{t}}$). At each iteration, the pixel compares whether the accumulated irradiance $\mathrm{PIX}$ at the photosensitive element is above a threshold $I_{\textrm{max}}$. Whenever the threshold is exceeded, the pixel is reset and a $6$-bits pixel-wise digital counter is incremented. To realize the mantissa operation, the gain of the pixel after each reset should be divided by $\alpha$. This means, it will take $\alpha$ times longer to integrate the same amount of light (assuming irradiance is constant) at the next iteration, thus realizing the log operation. This is summarized in the pseudo code of Algorithm~\ref{alg:our_method}.

\begin{algorithm}[t!]
\label{alg:our_method}
\newcommand{\mathrm{PIX}}{\mathrm{PIX}}
\newcommand{\mathrm{Frame}}{\mathrm{Frame}}
\newcommand{\mathrm{cnt}}{\mathrm{cnt}}
\newcommand{\mathrm{gain}}{\mathrm{gain}}
\caption{\small Pseudo code describing our implementation of MantissaCam. The for-loop highlighted in grey can be thought of as ``parallel-for" executed by all the pixels simultaneously. PIX is a variable representing the photosensitive element whose value is at any instant the integrated value of irradiance from the last pixel reset (one can alternatively think about every inner loop incrementing PIX by the irradiance $I_i\cdot\delta{t}$ where $\delta{t}$ is the time it took to perform the inner loop, c.f. blue comment).}
\DontPrintSemicolon
\SetKwRepeat{Repeat}{repeat for all frames}{}
\SetKw{Readout}{readout}
\Repeat{}{
    $n \gets 0$ \;
    \leavevmode\rlap{\hbox to \hsize{\color{lightgray!50}\leaders\hrule height .8\baselineskip depth .5ex\hfill}}\For{all pixels $i$}{
        $\mathrm{PIX}_i \gets 0$ \;
        $\mathrm{gain}_i \gets 1 $ \;
        \While{$n<N$}{
            \If{$\mathrm{PIX}_i \geq I_{\mathrm{max}}$}{
                $\mathrm{PIX}_i \gets 0$ \;
                $\mathrm{cnt}_i \gets \mathrm{cnt}_i+1$ \;
                $\mathrm{gain}_i \gets \frac{\mathrm{gain}_i}{\alpha}$ \;
            }
            \tcp{$\mathrm{PIX}_i \gets \mathrm{PIX}_i + \mathrm{gain}_i\, I_{i}\cdot\delta{t}$}
            $n\gets n+1$ \;
        }
        \Readout{winding number in $\mathrm{cnt}_i$} \;
        \Readout{mantissa in $\mathrm{PIX}_i$} \;
        }
}
\end{algorithm}
\subsection{Experimental Results}

We use SCAMP-5 to prototype MantissaCam by capturing a still life HDR scene, including a pear, a vase, and cloth under the direct illumination of a bright light bulb (see Fig.~\ref{fig:prototypephoto}). This sensor records grayscale images with a resolution of $256 \times 256$ pixels and, in our prototype implementation, emulates the procedure described in the previous section using the PE resources available in each pixel. For this experiment, our edge prediction and winding number prediction networks are re-trained using the same procedure and dataset described in Section~\ref{sec:methods:details}, using only grayscale images.

Figure~\ref{fig:scampresults} shows the tonemapped HDR scene (left) as well as a short  (center left) and a long LDR exposure (center). The dynamic range of this scene is far greater than that of the sensor. The mantissa-encoded LDR sensor image (center right) clearly shows the irradiance wraps in the brighter scene parts, around the light bulb and in the reflections off of the vase and the pear. The proposed reconstruction algorithm reconstructs the HDR scene at a reasonable quality, showing details in the filament and reflections.

\begin{figure}[t]
\begin{center}
   \includegraphics[trim=0cm 0cm 0cm 0cm, width=0.8\linewidth]{latex/figures/prototype.jpg}
\end{center}
   \caption{Prototype camera capturing the still life HDR scene.}
\label{fig:prototypephoto}
\end{figure}
\begin{figure*}[t]
\begin{center}
   \includegraphics[clip, trim=0cm 21cm 0cm 1.5cm, width=1\linewidth]{latex/figures/scampResults2.pdf}
\end{center}
   \caption{Experimental results. Using a programmable sensor, SCAMP-5, we obtain a ground truth HDR image of this scene from bracketed exposures (left and center). We prototype the proposed mantissa irradiance encoding scheme that wraps the irradiance into the LDR sensor image (center right) and reconstruct the HDR image (right) with the proposed algorithm.}
\label{fig:scampresults}
\end{figure*}

redpajama_set_name: RedPajamaArXiv


With Stan Lee's passing yesterday, one thought that has been on people's minds is what this would possibly mean for his cameos in future Marvel-based projects. Not exactly the prominent thought to have after someone passes, but I suppose it was inevitable. As far as 2019, you have a number of Marvel-related films coming out. You've got Captain Marvel, Avengers 4, Dark Phoenix, Spider-Man: Far From Home, and The New Mutants. What about after that, though? Well, Kevin Feige may have an idea.
The operative word being "may" because obviously you can't account for projects that aren't even in pre-production yet, but if you recall, at one point Marvel Studios indicated that it had filmed a batch of Stan Lee cameos. For which films? No idea yet. At the very least we know that his cameo for Avengers 4 is all said and done, so there's that. But what about beyond? And not just Marvel Studios, mind you, but Fox and Sony's own Marvel projects as well. So what did Kevin Feige have to say about future cameos?
And there you have it. Surprise. Obviously, at some point in time we'll reach a point where there won't be any Stan Lee cameos at all because, again, you have to account for many projects and adaptations that haven't even made it to the scripting phase yet, if at all. So there's nothing to extrapolate from Kevin Feige's words here regarding Lee's future cameos other than there being possible surprises. It's entirely possible that unused footage could be used as a cameo. We're already seeing that happen with the late Carrie Fisher and Leia's appearance in Star Wars: Episode IX.
As Marvel Studios stated that it had filmed a bunch of Stan Lee cameos already, perhaps some of those unused ones could be inserted into a future Marvel-related project. How and in what way? That remains to be seen.
In the meantime, what has been your favorite appearance by Stan Lee in a Marvel-related project thus far? And how do you believe he could potentially appear down the line? Let us know in the comments below.


text:
Ford introduces upgraded Focus line-up for 2010
New features for Ford Focus, new service initiatives
Ford Press July 30, 2010 13:51
The Ford Focus Deep Dive, Deep Drive
Ford launches the all new 2013 Focus
2016 Ford Focus 1.5L Titanium+
Special Agent Specs
Spied: 2018 Ford Focus sedan undergoing hot weather testing
2016 Ford Focus Hatchback Sport
2016 Ford Focus Hatchback Sport+
The Ford Focus Celebrity & Media Cup
This month, Ford Group Philippines announces the Focus as the latest in its line-up to receive updates for even more first-in-class technologies and features. The latest installment in the Ford Focus upgrades is also timed with the launch of an enhanced ownership experience with Ford's 24-Hour Parts Delivery Guarantee. The 2010 Ford Focus is now, more than ever, poised to delight customers with its performance, key class-leading vehicle features and after-sales service for the discerning Filipino motorist.
"The Focus just keeps on getting better and better-now, with welcomed improvements to both our product offering and service experience," says Steven Tan, Ford Group Philippines Vice President for Sales and Marketing. "There is no doubt that the Focus is one of the nameplates that truly embrace Ford's thrust to allow customers to appreciate the transformation in our brand and allow them to Feel the Difference. In the local market, there is no better time for us to translate this idea into real customer benefits and offerings that make a difference in the way our customers see, experience and feel the Ford brand than with the 2010 Ford Focus."
Earlier this year, the Ford Focus line-up was completed with the addition of the new Ford Focus TDCi 4-door sedan with six-speed PowerShift automatic transmission and the 1.8L Focus Trend sedan now with the convenience of an automatic transmission. The Ford Focus line-up was completed with a total of eight variants. The entry-level Ford Focus Style and Trend in 1.8L Duratec with manual transmissions, the Ford Focus Trend in 1.8L Duratec with automatic transmissions, available in both sedan and hatch, the Focus Ghia 4-door 2.0L Duratec AT, the Focus Sport Hatch 2.0L Duratec AT and finally, the top two variants with Ford's Six-speed Dual Clutch PowerShift technology, the Focus Ghia Sedan 2.0L Duratec AT and Focus Sport Hatch 2.0L Duratec AT.
With the latest evolution of the Ford Focus, the new product features further bolster the vehicle not only for better on-road performance but also driver and passenger comfort and convenience. "We have revamped our Ford Focus line-up with additional features to showcase the class-first technology and performance of this well-crafted vehicle," says Tan.
The Ford Focus Sport Hatch boasts of a new front bumper giving it a sportier edge, with body color matching Front Hood Bar and Fog Lamp Bezel. Its latest innovative offerings is through the 16-inch ST Design Wheels, specially designed for the ultimate comfort and smooth driving with the Ford Focus. With class-first features such as Rain Sensing Wipers, Auto On/Off Headlamps and Auto Dimming Rear View Mirror, this Ford Focus Sport is geared for optimum performance on the road with the comfort and convenience.
The Ford Focus Ghia TDCi sports the same innovative 16-inch ST Design Wheels, Rain Sensing Wipers, Auto On/Off Headlamps and Auto Dimming Rear View Mirror, while the Ford Focus Ghia Petrol's additional features include Rain Sensing Wipers, Auto On/Off Headlamps and Auto Dimming Rear View Mirror for ultimate performance.
The Ford Focus 1.8 L Trend sedan and hatch have been packed with 16" alloy wheels, upgraded from its past 15" wheels and now, with Front Passenger Airbag in addition to Driver Airbag, ensuring driver and passenger safety when on the road.
One of the Ford Focus' most distinctive features has always been the Ford PowerShift TDCi difference. The technology delivers the fuel efficiency and dynamics of a manual gearbox with the driving ease and full comfort of a premium automatic transmission, all because of its electronically-controlled dual-clutch technology to change gears. Not only does PowerShift allow you to tap into more power and control in its six-speed transmission, it also helps reduce fuel consumption by up to nine percent depending on the application.
Most recently, at the Ford Focus Coast to Coast Challenge 2 last May 4 to 7, 2010, the Ford Focus has set a new fuel economy record, garnering a total of 1,432.3km of total mileage as compared to the first challenge, a whopping106 kilometers difference.
The Ford Focus TDCi boasts of fuel economy but not at the expense of power with peak torque of 340Nm at 2,000rpm and maximum power of 136PS at 4,000rpm. This makes it not only powerful but also agile and responsive, with plenty of pulling power at low engine speeds for user-friendly everyday driving.
Excellent chassis control and steering precision give the Focus its on-road agility. When combined with the advanced and innovative technology of a DuraTorq turbo diesel engine and PowerShift automatic gearbox, the Focus TDCi is able to accelerate from zero to 100km/h in 9.6 seconds.
Jaguar Philippines launches the all new XJ
Jaguar Cars introduced the newest Jaguar XJ in ceremonies held last July 29, 2010 at Republiq Club at Resorts World Newport City.
All-new Nissan Sentra 200 Drives to Dealerships
The summer season may be over, but the streets are just starting to heat up as Nissan Motor Philippines, Inc. (NMPI) announced that the all-new Nissan Sentra will be available at all Nissan dealerships nationwide starting July 24.
Lincoln deletes some bling from 2020 Navigator
2020 Lincoln Navigator update adds more tech, offers 'bling deleted' monochromatic trio
Toyota PH recalls Rush: potholes, rough roads can trigger airbags
Toyota PH to send notices to affected Rush owners to have faulty airbag component replaced