<h2 dir="auto">Release 5.1.1</h2>
<p dir="auto">Hi community,</p>
<p dir="auto">After the release of 5.1.0 version, we have received a lot of feedback<br>
from users and developers. Now we plan to release the ShardingSphere 5.1.1<br>
version this week, which has made a lot of optimization and improvement for<br>
these feedback.</p>
<p dir="auto">Please refer to this release notes to confirm whether it contains the<br>
features you expect. If I miss anything, please remind me ASAP. If there<br>
are any suggestions, please feel free to tell us.</p>
<p dir="auto">Mail: <a href="https://lists.apache.org/thread/o9ys6db6mdnwlx0osd4nflw1rwbb3086" rel="nofollow">https://lists.apache.org/thread/o9ys6db6mdnwlx0osd4nflw1rwbb3086</a></p>
<h2 dir="auto">Release Notes</h2>
<h3 dir="auto">New Feature</h3>
<ol dir="auto">
<li>Kernel: support alter materialized view for PostgreSQL</li>
<li>Kernel: support declare for PostgreSQL</li>
<li>Kernel: support discard for PostgreSQL</li>
<li>Kernel: Add mode to parser to support $$ in PostgreSQL</li>
<li>Kernel: Support MySQL create tablespace statement parse</li>
<li>Scaling: Implement stop source writing and restore source writing</li>
<li>Scaling: Support partial tables scale-out</li>
<li>DistSQL: New DistSQL syntax: <code class="notranslate">SHOW UNUSED RESOURCES</code></li>
<li>Mode: Added persistent <code class="notranslate">XA Recovery Id</code> to Governance Center</li>
<li>Mode: Database discovery adds delayed master-slave delay function</li>
<li>Distributed Transaction: Add savepoint support for ShardingSphere proxy</li>
<li>Distributed Transaction: Support auto rollback when report exception in transaction block for PostgreSQL and openGauss</li>
<li>Distributed Transaction: Make it is easy to use with Narayana</li>
<li>Distributed Transaction: Add savepoint support for ShardingSphere-JDBC</li>
</ol>
<h3 dir="auto">Enhancement</h3>
<ol dir="auto">
<li>Kernel: Refactor kernel to improve performance</li>
<li>Proxy: Reduce Docker image size of ShardingSphere-Proxy</li>
<li>Proxy: ShardingSphere-Proxy supports set names statements</li>
<li>Proxy: ShardingSphere-Proxy MySQL supports multi statements</li>
<li>Scaling: Only one proxy node could do data consistency check in proxy cluster</li>
<li>Scaling: Replace scaling input and output config fields type from int to Integer</li>
<li>Scaling: Update MySQL checksum SQL</li>
<li>Scaling: Improve scaling job progress deletion in reset and progress check before starting job</li>
<li>Scaling: Improve <code class="notranslate">FinishCheckJob</code> data consistency check when target tables already have the same data as source tables</li>
<li>Scaling: Break scaling job ASAP when there is unsupported table since primary key</li>
<li>Scaling: Reuse <code class="notranslate">ClusterPersistRepository</code> of proxy in <code class="notranslate">PipelineAPIFactory</code></li>
<li>Scaling: Update jobId generation algorithm, and make it support idempotency</li>
<li>DistSQL: Support configuration data type and length when CREATE/ALTER ENCRYPT RULE</li>
<li>DistSQL: Unify the display results of <code class="notranslate">SHOW ALL VARIABLES</code> and <code class="notranslate">SHOW VARIABLE</code></li>
<li>DistSQL: Remove the effect of binding order when <code class="notranslate">DROP BINDING TABLE RULES</code></li>
<li>DistSQL: Add column <code class="notranslate">mode_type</code> in the result of <code class="notranslate">SHOW INSTANCE LIST</code></li>
<li>DistSQL: Add validation to the mode when <code class="notranslate">ENABLE/DISABLE INSTANCE</code></li>
<li>DistSQL: Check if the rule is in used when <code class="notranslate">DROP READWRITE_SPLITTING RULE</code></li>
<li>DistSQL: Check duplicate resource names when <code class="notranslate">CREATE READWRITE_SPLITTING RULE</code></li>
<li>DistSQL: Add column <code class="notranslate">delay_time</code> to the result of <code class="notranslate">SHOW READWRITE_SPLITTING READ RESOURCES</code></li>
<li>DistSQL: Support <code class="notranslate">IF EXISTS</code> when <code class="notranslate">DROP RULE</code></li>
<li>DistSQL: Optimize the prompt information of connection failure when <code class="notranslate">ADD/ALTER RESOURCE</code></li>
<li>Mode: Add schema-level global distributed locks</li>
<li>Mode: Add schema version number to support batch execution of DistSQL</li>
<li>Mode: Persistent metadata optimization in cluster mode</li>
<li>Mode: The database discovery add the <code class="notranslate">schemaName</code> identifier when create a JOB</li>
</ol>
<h3 dir="auto">Refactor</h3>
<ol dir="auto">
<li>Kernel: Refactor test case for encrypt</li>
<li>Kernel: Refactor metadata to support PostgreSQL database and schema</li>
<li>Scaling: Remove HikariCP dependency in pipeline modules</li>
<li>Mode: Refactor governance center storage node structure</li>
<li>Mode: Refactor governance center meta data structure</li>
<li>Mode: Adjust the database discovery MGR module to MySQL module</li>
</ol>
<h3 dir="auto">Bug Fix</h3>
<ol dir="auto">
<li>Kernel: Fix function with no parameter</li>
<li>Kernel: Fix <code class="notranslate">InsertValueContext.getValue</code> cast exception</li>
<li>Kernel: Fix aggregate distinct column error</li>
<li>Kernel: Fix NPE when rewrite parameter with schema</li>
<li>Kernel: Fix NPE caused by <code class="notranslate">GeneratedKeysResultSet</code> not return <code class="notranslate">columnName</code> in read-write splitting</li>
<li>Kernel: Fix show tables statement loses part of the single table</li>
<li>Kernel: Fix ModShardingAlgorithm wrong route result when exist same suffix table</li>
<li>Kernel: Fix sql parse error when contains key in assignment clause and optimize index parse</li>
<li>Kernel: Fix NumberFormatException when sharding algorithm config number props</li>
<li>Kernel: Fix wrong metadata when config single dataSource for read-write splitting</li>
<li>Kernel: Fix statement close exception when use <code class="notranslate">BatchPreparedStatementExecutor</code></li>
<li>Kernel: Fix rewrite lowercase logic when sql contains shorthand projection</li>
<li>Kernel: Fix NullPointerException when start up proxy with memory mode</li>
<li>Proxy: Fix literals may be replaced by mistake in PostgreSQL/openGauss protocol</li>
<li>Proxy: Fix ShardingSphere-Proxy PostgreSQL with multi-schema cannot be connected by PostgreSQL JDBC Driver 42.3.x</li>
<li>Proxy: Fix timestamp nanos inaccurate in ShardingSphere-Proxy MySQL</li>
<li>Proxy: Complete ShardingSphere-Proxy PostgreSQL codec for numeric in binary format</li>
<li>Proxy: Potential performance issue and risk of OOM in ShardingSphere-JDBC</li>
<li>Proxy: Fix Operation not allowed after ResultSet closed occasionally happens in ShardingSphere-Proxy MySQL</li>
<li>Proxy: Fix NPE causes by ShardingSphere-JDBC executeBatch without addBatch</li>
<li>Scaling: Fix failed or stopped job could not be started by DistSQL except restarting proxy</li>
<li>DistSQL: Fix parsing exception for inline expression when <code class="notranslate">CREATE SHARDING TABLE RULE</code></li>
<li>DistSQL: Fix parsing exception when password is keyword <code class="notranslate">password</code> in <code class="notranslate">ADD RESOURCE</code> statement</li>
<li>Mode: Fixed loss of compute nodes due to ZooKeeper session timeout</li>
<li>Mode: Fixed the case of the table name in the governance center</li>
<li>Mode: DistSQL enable disable instance refresh in-memory compute node status</li>
<li>Mode: Fixed database discovery unable to create Rule through DistSQL</li>
</ol>
<h2 dir="auto">Bug Report</h2>
<p dir="auto">[ERROR] 2021-07-05 15:43:33.881 [ShardingSphere-Scaling-execute-5] o.a.s.s.c.e.i.AbstractImporter - flush failed 0/3 times<br>
.<br>
java.sql.BatchUpdateException: 2Unknown exception: [INSERT INTO .... ON DUPLICATE KEY UPDATE can not support update for sh<br>
arding column.]<br>
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)<br>
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)<br>
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)<br>
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)<br>
at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)<br>
at com.mysql.jdbc.Util.getInstance(Util.java:408)<br>
at com.mysql.jdbc.SQLError.createBatchUpdateException(SQLError.java:1163)<br>
at com.mysql.jdbc.PreparedStatement.executeBatchedInserts(PreparedStatement.java:1632)<br>
at com.mysql.jdbc.PreparedStatement.executeBatchInternal(PreparedStatement.java:1298)<br>
at com.mysql.jdbc.StatementImpl.executeBatch(StatementImpl.java:970)<br>
at com.zaxxer.hikari.pool.ProxyStatement.executeBatch(ProxyStatement.java:128)<br>
at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeBatch(HikariProxyPreparedStatement.java)<br>
at org.apache.shardingsphere.scaling.core.executor.importer.AbstractImporter.executeBatchInsert(AbstractImporter.j<br>
ava:167)<br>
at org.apache.shardingsphere.scaling.core.executor.importer.AbstractImporter.doFlush(AbstractImporter.java:142)<br>
at org.apache.shardingsphere.scaling.core.executor.importer.AbstractImporter.tryFlush(AbstractImporter.java:127)<br>
at org.apache.shardingsphere.scaling.core.executor.importer.AbstractImporter.flushInternal(AbstractImporter.java:1<br>
18)<br>
at org.apache.shardingsphere.scaling.core.executor.importer.AbstractImporter.lambda$flush$2(AbstractImporter.java:<br>
109)<br>
at java.util.ArrayList.forEach(ArrayList.java:1259)<br>
at org.apache.shardingsphere.scaling.core.executor.importer.AbstractImporter.flush(AbstractImporter.java:104)<br>
at org.apache.shardingsphere.scaling.core.executor.importer.AbstractImporter.write(AbstractImporter.java:89)<br>
at org.apache.shardingsphere.scaling.core.executor.importer.AbstractImporter.start(AbstractImporter.java:81)<br>
at org.apache.shardingsphere.scaling.core.executor.AbstractScalingExecutor.run(AbstractScalingExecutor.java:47)<br>
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)<br>
at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(T<br>
rustedListenableFutureTask.java:125)<br>
at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:69)<br>
at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)<br>
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)<br>
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)<br>
at java.lang.Thread.run(Thread.java:748)<br>
Caused by: java.sql.SQLException: 2Unknown exception: [INSERT INTO .... ON DUPLICATE KEY UPDATE can not support update for<br>
sharding column.]<br>
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:965)<br>
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3978)<br>
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3914)<br>
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2530)<br>
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2683)<br>
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2495)<br>
at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1903)<br>
at com.mysql.jdbc.PreparedStatement.executeUpdateInternal(PreparedStatement.java:2124)<br>
at com.mysql.jdbc.PreparedStatement.executeUpdateInternal(PreparedStatement.java:2058)<br>
at com.mysql.jdbc.PreparedStatement.executeLargeUpdate(PreparedStatement.java:5158)<br>
at com.mysql.jdbc.PreparedStatement.executeBatchedInserts(PreparedStatement.java:1593)<br>
... 21 common frames omitted</p>
<h3 dir="auto">Which version of ShardingSphere did you use?</h3>
<p dir="auto">apache-shardingsphere-5.0.0-beta-shardingsphere-scaling-bin.tar.gz</p>
<h3 dir="auto">Which project did you use? ShardingSphere-JDBC or ShardingSphere-Proxy?</h3>
<p dir="auto">I use scaling5.0 migration data to ShardingSphere-Proxy4.1.0</p>
<h3 dir="auto">Expected behavior</h3>
<h3 dir="auto">Actual behavior</h3>
<h3 dir="auto">Reason analyze (If you can)</h3>
<h3 dir="auto">Steps to reproduce the behavior, such as: SQL to execute, sharding rule configuration, when exception occur etc.</h3>
<h3 dir="auto">Example codes for reproduce this issue (such as a github link).</h3>