Ever since Rick Nash was formally put on the block last season, we have had quite a bit of discussion on the topic here at BSB. Interestingly, it seems our readers (and writers, for that matter) are pretty evenly divided in the pro or anti-Nash camps. This is understandable given his large and lengthy contract, his (seemingly) declining production and his superstar status on a basement dweller.
If you read here regularly, you are aware that Dave and I don't necessarily see eye to eye on a potential Nash acquisition. Due to our never ending quest for objective examination, I asked Dave to give me an assist on some advanced stats analysis to dig a little deeper into the merits of rolling the dice on Mr. Nash. Since Nash has experienced a production decline in each year since his 40 goal '08-'09 campaign, I wanted to know if this was due to actual decline or if the decrease in surrounding talent was more to blame.
Note by Dave: I admittedly don't watch much of the Columbus Blue Jackets, so there's a solid chance I'm off on this a bit.
2009-2010: The only year in this time frame that Nash had legitimate help. R.J. Umberger, Antoine Vermette, and Kristian Huselius were tremendous assets offensively for CBJ. Nash was clearly the best, but he had a clear-cut points in the standings advantage over them. Interestingly enough though, Nash's RCorsi in comparison with the rest suggest he did all of the work in maintaining possession, and the rest just benefited from him. This is the only year where Nash's RCorsi was better than everyone else.
2010-2011: Umberger was Nash's greatest asset this season, but the QoC suggests they didn't play together frequently. QoC suggests that Vermette got the majority of time with Nash, where his 7.8 RCorsi was a big help. Nash himself had a good puck possession year at well. Nash again was worth more than a full point in the standings than Vermette and Derek Brassard combined.
2011-2012: It's very clear that the opposition sent their best players out against Nash to shut him down. His .058 QoC was the highest he's ever faced. Vinny Prospal was a key addition to the BJs, his RCorsi is through the roof and it appears that he helped make that line go in the way that Hagelin was a good fit for the Rangers top line. Even though Prospal was a great fit, Nash was still worth more than a point more in the standings than Prospal, and Nash was worth more in the standings than both Umberger and Brassard combined.
As you can see from Dave's analysis, it seems as if Nash does indeed perform better with a better supporting cast. Obviously, this is not a new concept for professional athletes, but considering the context of the team itself, it makes perfect sense. The year before Dave's research began, the Jackets finished with 92 points and made the playoffs for the only time in franchise history. The three years incorporated in Dave's work saw the point totals and most types of production fall of a cliff in Ohio, the only question is, why?
You could obviously place some of the blame on the Captain and highest paid player in Nash. Conversely, you could blame quite a bit of the decline on Steve Mason's horrendous run. Bad trades/free agent signings also played a part. Given the difference in line mates that Nash had with him in those subsequent years, I think it would be extremely difficult to expect him to keep up a 40 goal pace in those circumstances. Don't even start on plus/minus numbers, they mean even less than usual when it comes to the Jackets.
To start the season, if the Rangers acquired Nash, he would likely play with Richards and Hagelin. Both of those players would be an upgrade on the dearth of talent Nash has in Columbus. Since his decline in production seemingly coincides with a downgrade in peripheral talent and significant drop in success of the organization as a whole, I tend to believe a rebound in production would be very likely.
Will Nash be a 40-goal guy again in a system like the one Torts employs? It's hard to tell for sure, but there are no signs of physical decline in Nash's game, only downgraded personnel around him. I'll take that scenario over a point producer who all of a sudden has a down year for a contending team.
Now, the contract. Zach Parise just made Rick Nash's contract seem like a bargain. Obviously, $7.8 million per season is not chump change, especially in a salary cap context. The term is so much more favorable in this case that it really is a no-brainer for me at this point.
Parise generally tends to out-produce Nash, let's say to the tune of about 10 points per season. Is that worth an extra seven years on a contract? If both were free agents and you could have Nash for 6/$47 million or Parise for 13/$98 million, is it even a debate? You are out from under Nash's deal at age 34 versus 40+ for Parise.
In the context of the Rangers current contracts it makes sense as well. Gaborik will be a free agent two seasons from now at the age of 32. If the Rangers decide to let him walk, Nash's hit essentially replaces Gaborik's. It also gives the Rangers more freedom to play hardball on a new contract with Gaborik.
All in all, while I think Bobby Ryan is a better value, Sather clearly covets a physical power forward in Nash's mold. I personally would love to see #61 on Broadway and I think Parise's new contract has shown how expensive this type of production is becoming. What do you guys think? Has this analysis done anything to change your Nash position? Sound off in the comments.
*If the Rangers actually acquire Nash, we will dig deeper into the metrics and systems to find out exactly how his game fits in with NY.
** Also, this entire analysis is predicated on a reasonable trade package to acquire Nash's services.
Just a point that seems to be overlooked.
Ryan is slated as the better trade since his cap hit and length is better than Nash's.
Has anyone thought past the next 3 years when Ryan will be a UFA? What type of contract will he command? And what about the possibility of him walking away to sign with another team of his choosing?
There is a benefit of Nash being locked up for so long. Its the benefit of Nash being locked up and not hitting Free Agency after we give up assets to get him.
Because Gaborik is UFA in 2 years, if Ryan works out well NYR can choose to pay Ryan Gaborik-class money, which he may or may not deserve. Also, in 3 years it's impossible to say what kind of money players like Callahan, McD, MDZ, Kreider, Hagelin, Miller, etc will cost, not to mention having just about 7 million locked up in Richards. Hence, a little breathing room and a chance to re-evaluate in 3 years is, in my opinion, much more important.
Doan is a bit of a stretch there because he wants to go back, but the PHX ownership issue is forcing his hand.
I would rather have Nash for this very reason… among others. We also know that in order to get Ryan, we need to move Stepan. That might not be the case with Nash.
Stay the course. We don't need either of them.
I would take Ryan. But Nash. It would be like getting Roberto luongo. Good player but overpaid. I also cringe at dropping 20+ mil on a first line.
Nash is only slightly overpaid, just like the majority of players in the league.
The first line is where you are SUPPOSED to spend all your money. Elite scoring talent is how you win!
You put Nash on a line that requires teams to defend all three forwards and he is a 35 goal 70 point machine for the remainder of his contract. In Columbus a center, wing and defender would guard/block him cause rest of team was worthless. Quiet giant, great off ice guy, just wants to win.
I agree Bill. I think Nash would be a huge contributor in NY, and after showing years of loyalty to the Jackets, he would get his chance to win.
Heres why Im pro Nash. He brings a skill set this team does not have, someone who can create on his own. I look at a player like Pavel Datsyuk on Detroit, similar skill set and turns 34 this month (the age Nash contract is up). Datsyuk is 5'11 to Nash who is 6'4. Nash has more goals in less games. By more goals I mean more overal, pp/sh/gwgs. Nash has scored 30 7 out of 9 seasons while Datsyuk has scored 30, 3 of 10. While Datsyuk may have more assist cant even compare teams. Also, Nash best years were under Ken Hitchcock whos a pretty demanding coach, just like Torts.
Who would you give up?
Id go Hagelin, Dubi, Erixon, and either Thomas/Miller/Anisimov with a first.
What do you do while Gaborik's hurt?
That's a lot to give up….and you actually give up a lot in the process. That's the farm, the d – depth and our left side….and dubi is great on draws.
Also remember that Sauer is out, McIlrath is out, Gabby is out. Erixson has to make the team next year or its bickel time.
Howson's demands are still too high.
To you and John…I would assume that this deal, like every other deal involving a superstar, would have pieces coming back with Nash.
Id like to see them go out and get Peter Mueller and see what he brings to the table if he stays healthy he could bring some scoring to the third line when Gaborik returns. If not they can survive with Nash, Richards and crew.
Thanks Dave, have read the blogs for a couple months now, first time ive commented.
Keep em coming. The more discussion the better!
I agree that the trade proposal is solid. Just a fine point though… if Dubi & Hagelin go, then Miller will not. He is really the only guy in the system that could replace what Dubi & Hags bring over the next year or two. It would be like trading DZ and Erixon in the same offer… it just doesn't make sense because Erixon would replace DZ.
I would include Kreider & Dubinsky and maybe a mid-tier player from the farms. Kreider has lots of potential, so use it as a selling point.
Interesting logic. Trade two players who have a realistic projection to score more goals combined than Nash at half the price.
You can't be serious, can you? Kreider is a deal breaker, the kid has a ton of upside to him, you can't be thinking straight!
Kreider has 10 games under his belt. Not a proven career. The league is full of guys with "potential", but the money players do it year after year. Nash is money, Kreider is vapor. Tell me that Kreider in a full year with NYR will score 30 and then tell me if you are being realistic. He has a ton of upside, so did Hugh Jessiman.
Comparing kreider to jessiman is absurd, your logic is completely hearsay. kreider has the most potential of all our prospects. If kreider was deal breaker before playing a game he is off the table now after playing just 10.
Like Blueshirt in Paris wrote, trade 2 players who have realistic projection to score more goals combined, at half the price, not a good idea.
Kreider is vapor, I'll take my chances any day with vapor. As for Hugh Jessiman, how many trades, and or signings, over the years really worked out for this team. After you think it through, then get back to me?
Krieder = vapor. This guy makes me laugh. Nash is indeed a stud, but CK is too much of a sure thing to give up.
Nash would look good in Ranger blue, but at what cost. So would Towes, Ovie, Kovie, Crosby, Malkin, etc!
Let's get serious, we can trade for Nash, and give away the entire team in the process, then Nash would be in the same shape he is in now, on a losing team.
Make a decent offer, then if they refuse, try to make a deal fo Ryan. If that can't be worked out, play the kids, and let them all sit with the Nashs, Ryans, etc., and continue to lose!
Crazy how divisive a topic this is. I love me some Rick Nash (obviously), but Kreider is a deal breaker for me as well. I would probably be willing to give up more than most because I don't view the contract as negatively, but for me it probably stops at Dubi, Thomas, McIlrath/Erixon and a 1st.
I agree there is no way im including Kreider in any trade. Yes he does have potential and yes Hugh jessiman had potential, but did u watch the playoffs? The kid was that talented that the Rangers burned a year off his contract to have him for the playoffs. He's showed he can do it, in the playoffs against some of the better teams in the East. You can't trade someone like that IMzo.
Move Kreider while he still has value. Nash is a sure thing. Don't make any deals and stand your ground Slats. Then, Ranger fans, cry in December when you continue to lose games 2-1 with a 3rd & 4th line that mucks in the corner but CANNOT PUT THE PUCK IN THE NET. All of the players we signed still won't help our breakout or score 20+. You want to see what Kreider can do, fine, but don't complain when he turns into Scott Gomez!
I've been watching Kreider for a long time, know him and know his game, and while I don't think he'll be a top-10 scorer in the league he's going to be a force. More than anything because of his winning attitude and drive to improve. Granted that doesn't translate directly to success, but the kid's talented too, and is an awesome skater. Put all those things together and there's no way you get a Gomez.
Sounds like SalMerc is just trying to rile people up, because I can't see how anyone would possibly think that Krieder is the next coming of Scott Gomez. Kinda silly actually.
Nah, I just think he's recognizes a limited window to win. I may not agree with him, but he's not the only one who is thinking that trading Kreider to make a run is an option.
Need 1 more roster player there to make it a deal Howson would bite at.
Would have to assume another player comes back with Nash to fill holes left by the trade.
Add Artem, get mueller and take away the 1st?
Still need the 1st round pick there, probably remove a prospect if anything.
I think one really important question no one's asking is: How will Nash's style of play mesh with Richards' and Gaborik's? Richards is a distributor, for sure, but it's clear that he and Nash both like to carry the puck in on the rush as often as possible. Similarly, Gaborik is a player who typically needs to have the puck on his stick to be effective. In a perfect world, you have Nash going to the hard areas, opening up the ice for Richards to make a bang-bang pass to Gaborik who can one-time it, or even a situation where Richards is passing to Nash who's taking a booming shot and Gaborik is getting in behind the D for a rebound (or vice versa). But how often have we seen that kind of setup in Torts' system? His emphasis on board play seems to stifle offensive creativity somewhat, meaning that–in my opinion–Nash is a better fit, with his bigger, stronger style of play, than even Gaborik. On top of that, Stepan and Gaborik seemed to have better chemistry together than Richards and Gaborik for the majority of the season, and while that may change I think it's better not to include Stepan in a trade in case Richards and Gaborik's chemistry doesn't continue to grow–or, more importantly, in case a different top-3 RW is playing so well on the top line that you don't want to move usurp him for Gaborik immediately.
And when/if Gaborik gets back to 100% ability, or if Kreider/Hags isn't playing well on the #1, you switch them.
But I guess my point is: I don't think you can afford to trade Stepan for Nash, and I don't think you can (or should!) automatically pencil in a line of Nash-Richards-Gaborik, because nothing wins cups better than scoring depth. Also I think Kreider will have a huge rookie season, but that's a different story.
There was a pretty good article recentley, I forget where and by whom… I think Ray Ferraro was quoted in it.
Basically it said that Nash still has a lot of 'junior' in his game. Trying to do too much by himself which is a product of playing in CBJ.
They went on to say that he needs to stop carrying the puck and drive to the net more without the puck and rely/trust in his center to get him the puck.
Nash carried the puck in CBJ bc no one else there could do it. With Richards, MDZ, and McD, there are enough puck carriers to allow Nash to do his thing.
Why all the need for Nash? Is it because people think we can win a cup with him on the team?
Do we think that addition by subtraction is the way to go?
It took 10 years to get to this point, where we have great potential on the team, a farm system with decent prospects. Speedy players, with loads of upside.
We built from the ground up…not just drafting but trades to get players that are young…and taught them.
7 out of 12 forwards groomed within. Gem of the NHL…you can't deny that.
In the wings, Borque, Thomas, Miller, St.Croix, Forgaty, Erixson….etc.
Why trade for Nash, he is not a Messier, or a Yzerman, Gretzky, etc. He is good, I don't doubt that, what I doubt is that we will get a good deal on him. Or better yet, we won't get a fair deal on him.
If that's the case we need to move on, I think Semin would be a much better fit for two reasons, 1 we can sign him for a shorter contract, 2 We don't have to give as much up to add him to the roster, cancer or not he is much more cost effective and pushes us toward our goal.
I feel the playoffs is your proof that you need a sniper/scorer/skater. We played great defense in front of Hank, but almost every game was close. Did you worry when Kovulchuck or Parise were on the ice? I did. Not sure our opponents feared the NYR powerplay, our ability to skate out of the zone or any one player (other than Gabby). Give me a goal scorer. We have lots of unselfish players, but few who scare the opposition when they are on the ice. Remember that Kovulchuk was considered a cancer before he joined NJD. Maybe Semin with his baggage can fit the bill without giving up Kreider/Stepan, but be prepared to only see him on power-plays in the 3rd period of one goal games.
i think the playoffs proved that stanley cup winners need 3rd and 4th line scoring and excellent 2 way play. i agree the powerplay needs to improve dramatically, but the best way to do that is start throwing shots at the net with bodies in front, too many open lane shots.
i agree semin is sketchy, but a one year deal in a year that feels like itll be a step back year (5-7th in the conference, second round exit) but will put another year under their belts and a playoff heartbreak in the back of their minds. mine as well try him.
though i like the thinking of a couple people to see what mueller can do, hes got talent and a 1-2 year contract doesnt hurt even if hes got concussion issues.
lastly, im a big fan of Nash and i would love to see him in a jersey, but hes just not worth the price both in assets and cap space. but give me a reasonable deal (dubi, 1st, and maybe 2 of miller/anisimov/exixon/another 1st im in.
p.s. do you thing columbus would accept a heavy draft pick package vs prospects and roster players if they are really in for the rebuild?
You are completely missing a major cog of the "build from within" method of developing your team. When you get the point where the CUP is in reach, you use your assets to TRADE for players that will help get you there or sign free agents that will help. In reality, when you say "stay the course", you are pretty much guaranteeing that this team has a much lower chance of winning a CUP in the next decade. The players they have in the pipeline and on the team are good, but there is still a major lack of elite scoring talent. We have many players that would be 2nd/3rd liners on on other teams. That needs to change, which was as clear as day in the playoffs.
Semin? Haven't you read and heard enough about this guy to understand that he would be the worst possible fit for this team?
Then maybe we need to change the way we play to win the cup.
I don't want to win one cup and wait 15 years to restock the system.
We have the assets to win the cup….if gabby was healthy would we even be having this conversation? We would have been in the finals. One player does not make a cup contender. And A Nash trade could win us a cup….but so would doing other things.
I don't know, i am not drinking the koolaid on this one. I think there are other options and the asking price is too high.
I do agree that if the price is too high, you don't make the deal. But my point is that moving a glut of 2nd/3rd liners (Dubi, AA, even Hags) doesn't represent a high price. Love Hags, but he will never be in the same stratosphere as Nash.
This looks/sounds to me as a classic under-value/over-value situation where Rangers fans are under on Nash and Over on their own.
Lets try and stay objective if we can and realize that no matter what out personal feeling are about Rick he is still CBJ's franchise player, their captain, their chance to start a rebuild and one of the best players in the league.
Yeah, as a Rangers fan I would love to only part with Dubi, Thomas and scraps but thats about as realistic as dry water.
Dubi had 2 decent seasons and last years debacle, Erixon couldnt really hack it at the NHL level and Hagelin has some moments but no one can say for certain he is a top 6 forward going forward, which leaves us with Thomas (never stepped foot in an NHL game) who is undersized and a 1st round pick which will be close to a second rounder.
IF the rumor of Stepan, Dubi, and 1st. Would get it done, I would think Sather would have to let go of Stepan to get the deal done, and or maybe the next 2 1st round picks.
Keep in mind Stepan is a 2nd line center that scored 50pts. His upside is as strong as it gets as a Center in the NHL. And the extra 1'st rounders have to help a building team. CBJ realise they have to take Dubi in the deal do to Nash's contract.
A few Prospects like a Thomas or Zucc's rights, would still be very good prospects for the CBJ, and a very good opportunity for Zucc and Thomas to make the NHL. But by no means are they scraps. Look at what CBJ put on the ice last year? That means their farm has to be even worse.
*Therefore rumor-monger says: Boyle going to CBJ for Nash.
An Unbelievable source is just that = unbelievable!!!
Another day without a bona-fide scorer on the roster. Yea, don't worry guys, Artie & Boyle will put 25 in each this year and our power-play will magically jump from 23rd to 3rd. Hank will be stellar again and our monster off-season will make Prust all but a trivia question. Pass the Kool-aid.


text:
Bills seek to strengthen Guam's law against animal cruelty
The court's controversial decision to exonerate a pet killer in Yigo over legal technicality has prompted senators to file bills that would put more teeth to Guam's anti-animal cruelty law.
Sen. Louise Muña introduced Bill 48-35, which categorizes the crime of the "unlawful killing" a pet as "animal abuse in the first degree."
If enacted into law, this change would have the effect of making the crime a felony regardless of how the perpetrator kills or seriously injures the pet, according to the bill.
Superior Court Judge Michael Bordallo last week dismissed all felony charges against Gerald Wayne Cruz II, who was was indicted in a series of dog and cat shootings in Yigo last year, arguing that the case failed to establish intentional acts of cruelty.
"It is difficult to think of a less cruel way to kill an animal than death by gunshot," Bordallo said, ruling on the test case of animal cruelty on Guam. The judge said the indictment accused Cruz "only of shooting a dog or shooting a cat" and that his action did not establish intention to inflict "mental and physical sufferings on the animals."
Bill 48-35 would make the unlawful killing of another person's pet a third degree felony. The bill also raises the penalty to a second degree penalty if the perpetrator commits multiple offenses or is a repeat offender.
"Most pet owners to don't view themselves as 'owners' but rather as 'parents' of their pets," Muña said. "Nearly all states have repealed firing squad executions laws and, in doing so, recognized it as a 'cruel and unusual' form of punishment. We also, must recognize that the unlawful shooting of a pet is an extremely cruel and barbaric act and I hope that prosecutors appeal this ruling."
Sen. Sabina Perez's Bill 49-35 seeks to clarify that "animal abuse in the first degree includes the act of cruelly and maliciously causing the death of an animal."
The bill is co-authored by Sens. James Moylan, Joe San Agustin, and Amanda Shelton.
Guam law currently stipulates a third-degree felony for anyone who "cruelly causes the death" of an animal. The judge dismissing the charges cited this section, narrowly interpreting the term "cruelly" to only encompass whether an animal suffers from prolonged pain.
"It's deeply disturbing that an individual can now trespass onto private property, shoot a family's dog, and only receive a slap on the wrist," Perez said.
"It was never the intent of the law to allow something like this to happen," she added.
In order to prevent similar future rulings, Bill 49-35 amends the law to apply a third-degree felony charge to anyone who "cruelly or maliciously causes the death" of an animal. With this change, all cruel or ill-intentioned killings of animals will be considered a felony. Hunting game, self-defense, and other legal exemptions will continue to exist in statute.
"The recent court ruling creates a disturbing legal precedent," Perez said. "Bill 49-35 makes clear the original intent of the law, which is to protect all animals from senseless cruelty."

redpajama_set_name: RedPajamaCommonCrawl


text:
/*
	Package whisper implements Graphite's Whisper database format
*/
package whisper

import (
	"bytes"
	"encoding/binary"
	"errors"
	"fmt"
	"math"
	"os"
	"regexp"
	"runtime/debug"
	"sort"
	"strconv"
	"strings"
	"syscall"
	"time"
)

const (
	// size constants
	ByteSize        = 1
	IntSize         = 4
	FloatSize       = 4
	Float64Size     = 8
	PointSize       = 12
	MetadataSize    = 16
	ArchiveInfoSize = 12
)

const (
	Seconds = 1
	Minutes = 60
	Hours   = 3600
	Days    = 86400
	Weeks   = 86400 * 7
	Years   = 86400 * 365
)

const (
	classicHeaderAggregationOffset = 0
	classicHeaderXFFOffset         = IntSize * 2
)

// Note: 4 bytes long in Whisper Header, 1 byte long in Archive Header
type AggregationMethod int

const Unknown AggregationMethod = -1
const (
	Average AggregationMethod = iota + 1
	Sum
	Last
	Max
	Min
	First

	Mix        // only used in whisper header
	Percentile // only used in archive header
)

func (am AggregationMethod) String() string {
	switch am {
	case Average:
		return "average"
	case Sum:
		return "sum"
	case First:
		return "first"
	case Last:
		return "last"
	case Max:
		return "max"
	case Min:
		return "min"
	case Mix:
		return "mix"
	case Percentile:
		return "percentile"
	}
	return fmt.Sprintf("%d", am)
}

func ParseAggregationMethod(am string) AggregationMethod {
	switch strings.ToLower(am) {
	case "average", "avg":
		return Average
	case "sum":
		return Sum
	case "first":
		return First
	case "last":
		return Last
	case "max":
		return Max
	case "min":
		return Min
	case "mix":
		return Mix
	case "percentile":
		return Percentile
	}
	return Unknown
}

type Options struct {
	Sparse bool
	FLock  bool

	Compressed bool
	// It's a hint, used if the retention is big enough, more in
	// Retention.calculateSuitablePointsPerBlock
	PointsPerBlock int
	PointSize      float32
	InMemory       bool
	OpenFileFlag   *int

	MixAggregationSpecs        []MixAggregationSpec
	MixAvgCompressedPointSizes map[int][]float32

	SIMV bool // single interval multiple values

	IgnoreNowOnWrite bool
}

type MixAggregationSpec struct {
	Method     AggregationMethod
	Percentile float32
}

// a simple file interface, mainly used for testing and migration.
type file interface {
	Seek(offset int64, whence int) (ret int64, err error)
	Fd() uintptr
	ReadAt(b []byte, off int64) (n int, err error)
	WriteAt(b []byte, off int64) (n int, err error)
	Read(b []byte) (n int, err error)
	Name() string
	Close() error
	Write(b []byte) (n int, err error)
	Truncate(size int64) error
}

/*
	Represents a Whisper database file.
*/
type Whisper struct {
	// file *os.File
	file file

	// Metadata
	aggregationMethod AggregationMethod
	maxRetention      int
	xFilesFactor      float32
	archives          []*archiveInfo

	compressed             bool
	compVersion            uint8
	pointsPerBlock         int
	avgCompressedPointSize float32
	crc32                  uint32

	opts     *Options
	Extended bool

	// TODO: improve
	NonFatalErrors []error

	discardedPointsAtOpen uint32
}

/*
  A retention level.

  Retention levels describe a given archive in the database. How detailed it is and how far back
  it records.
*/
type Retention struct {
	secondsPerPoint int
	numberOfPoints  int

	// for compressed whisper (internal)
	avgCompressedPointSize float32
	blockCount             int
}

/*
  Describes a time series in a file.

  The only addition this type has over a Retention is the offset at which it exists within the
  whisper file.
*/
type archiveInfo struct {
	Retention
	offset int

	next    *archiveInfo
	whisper *Whisper

	// NOTE: buffer design deprecated for v2 and mix
	//
	// why having buffer:
	//
	// original reasons:
	// 	1. less file writes per point
	// 	2. less file reads & no decompressions on propagation
	//
	// necessary reasons:
	//  cwhisper doesn't expect data points coming in randomly, having a  buffer
	//  allows it to tolerate data points with different timestamp coming in
	//  non-increasing order for whatever reasons. But only the first/base archive
	//  is necessary to have it, so it's possible to optimize away buffers in lower
	//  archives.
	buffer     []byte
	bufferSize int // dynamically calculated in Whisper.initMetaInfo

	blockRanges []blockRange // TODO: remove: sorted by start
	blockSize   int
	cblock      blockInfo // mostly for quick block write

	aggregationSpec *MixAggregationSpec

	stats struct {
		// interval and value stats are not saved on disk because they could be
		// regenerated by scanning blocks
		interval struct {
			len1, len9, len12, len16, len36 uint32
		}
		value struct {
			same, sameLen, variedLen uint32
		}

		extended uint32

		discard struct {
			oldInterval uint32
		}
	}
}

type blockInfo struct {
	index          int
	crc32          uint32
	p0, pn1, pn2   dataPoint // pn1/pn2: points at len(block_points) - 1/2
	lastByte       byte
	lastByteOffset int
	lastByteBitPos int
	count          int
}

type blockRange struct {
	index      int
	start, end int // start and end timestamps
	count      int // should be named as index
	crc32      uint32
}

func unitMultiplier(s string) (int, error) {
	switch {
	case strings.HasPrefix(s, "s"):
		return Seconds, nil
	case strings.HasPrefix(s, "m"):
		return Minutes, nil
	case strings.HasPrefix(s, "h"):
		return Hours, nil
	case strings.HasPrefix(s, "d"):
		return Days, nil
	case strings.HasPrefix(s, "w"):
		return Weeks, nil
	case strings.HasPrefix(s, "y"):
		return Years, nil
	}
	return 0, fmt.Errorf("invalid unit multiplier [%v]", s)
}

var retentionRegexp *regexp.Regexp = regexp.MustCompile(`^(\d+)([smhdwy]+)$`)

func parseRetentionPart(retentionPart string) (int, error) {
	part, err := strconv.ParseInt(retentionPart, 10, 32)
	if err == nil {
		return int(part), nil
	}
	if !retentionRegexp.MatchString(retentionPart) {
		return 0, fmt.Errorf("%v", retentionPart)
	}
	matches := retentionRegexp.FindStringSubmatch(retentionPart)
	value, err := strconv.ParseInt(matches[1], 10, 32)
	if err != nil {
		panic(fmt.Sprintf("Regex on %v is borked, %v cannot be parsed as int", retentionPart, matches[1]))
	}
	multiplier, err := unitMultiplier(matches[2])
	return multiplier * int(value), err
}

/*
  Parse a retention definition as you would find in the storage-schemas.conf of a Carbon install.
  Note that this only parses a single retention definition, if you have multiple definitions (separated by a comma)
  you will have to split them yourself.

  ParseRetentionDef("10s:14d") Retention{10, 120960}

  See: http://graphite.readthedocs.org/en/1.0/config-carbon.html#storage-schemas-conf
*/
func ParseRetentionDef(retentionDef string) (*Retention, error) {
	parts := strings.Split(retentionDef, ":")
	if len(parts) != 2 {
		return nil, fmt.Errorf("not enough parts in retentionDef [%v]", retentionDef)
	}
	precision, err := parseRetentionPart(parts[0])
	if err != nil {
		return nil, fmt.Errorf("failed to parse precision: %v", err)
	}

	points, err := parseRetentionPart(parts[1])
	if err != nil {
		return nil, fmt.Errorf("failed to parse points: %v", err)
	}
	points /= precision

	return &Retention{secondsPerPoint: precision, numberOfPoints: points}, err
}

func ParseRetentionDefs(retentionDefs string) (Retentions, error) {
	retentions := make(Retentions, 0)
	for _, retentionDef := range strings.Split(retentionDefs, ",") {
		retention, err := ParseRetentionDef(retentionDef)
		if err != nil {
			return nil, err
		}
		retentions = append(retentions, retention)
	}
	return retentions, nil
}

func MustParseRetentionDefs(retentionDefs string) Retentions {
	rets, err := ParseRetentionDefs(retentionDefs)
	if err != nil {
		panic(err)
	}
	return rets
}

// Wrappers for whisper.file operations
func (whisper *Whisper) fileWriteAt(b []byte, off int64) error {
	_, err := whisper.file.WriteAt(b, off)
	return err
}

// Wrappers for file.ReadAt operations
func (whisper *Whisper) fileReadAt(b []byte, off int64) error {
	_, err := whisper.file.ReadAt(b, off)
	return err
}

/*
	Create a new Whisper database file and write it's header.
*/
func Create(path string, retentions Retentions, aggregationMethod AggregationMethod, xFilesFactor float32) (whisper *Whisper, err error) {
	return CreateWithOptions(path, retentions, aggregationMethod, xFilesFactor, &Options{
		Sparse: false,
		FLock:  false,
	})
}

// CreateWithOptions is more customizable create function
//
// avgCompressedPointSize specification order:
// 		Options.PointSize < Retention.avgCompressedPointSize < Options.MixAggregationSpecs.AvgCompressedPointSize
func CreateWithOptions(path string, retentions Retentions, aggregationMethod AggregationMethod, xFilesFactor float32, options *Options) (whisper *Whisper, err error) {
	if options == nil {
		options = &Options{}
	}
	if aggregationMethod == Mix && !options.Compressed {
		return nil, errors.New("mix aggregation method is currently supported only for compressed format")
	}
	sort.Sort(retentionsByPrecision{retentions})
	if err = validateRetentions(retentions); err != nil {
		return nil, err
	}
	_, err = os.Stat(path)
	if err == nil {
		return nil, os.ErrExist
	}
	var file file
	if options.InMemory {
		file = newMemFile(path)
		err = nil
	} else {
		file, err = os.Create(path)
	}
	if err != nil {
		return nil, err
	}

	if options.FLock && !options.InMemory {
		if err = syscall.Flock(int(file.Fd()), syscall.LOCK_EX); err != nil {
			file.Close()
			return nil, err
		}
	}
	if options.PointSize == 0 {
		options.PointSize = avgCompressedPointSize
	}
	if options.PointsPerBlock == 0 {
		options.PointsPerBlock = DefaultPointsPerBlock
	}

	whisper = new(Whisper)

	// Set the metadata
	whisper.file = file
	whisper.aggregationMethod = aggregationMethod
	whisper.xFilesFactor = xFilesFactor
	whisper.opts = options

	whisper.compressed = options.Compressed
	whisper.compVersion = 1
	whisper.pointsPerBlock = options.PointsPerBlock
	whisper.avgCompressedPointSize = options.PointSize
	for _, retention := range retentions {
		if retention.MaxRetention() > whisper.maxRetention {
			whisper.maxRetention = retention.MaxRetention()
		}
	}

	// Set the archive info
	for i, retention := range retentions {
		archive := &archiveInfo{Retention: *retention}

		if archive.avgCompressedPointSize == 0 {
			archive.avgCompressedPointSize = whisper.avgCompressedPointSize
		}
		if archive.blockCount == 0 {
			archive.blockCount = whisper.blockCount(archive)
		}

		if whisper.aggregationMethod == Mix && i > 0 {
			for i, spec := range options.MixAggregationSpecs {
				narchive := *archive
				narchive.aggregationSpec = &MixAggregationSpec{Method: spec.Method, Percentile: spec.Percentile}
				ssp := narchive.secondsPerPoint
				sindex := i % len(options.MixAggregationSpecs)
				if msizes := options.MixAvgCompressedPointSizes; msizes != nil &&
					msizes[ssp] != nil &&
					isGoodFloat32(msizes[ssp][sindex]) {
					narchive.avgCompressedPointSize = msizes[ssp][sindex]
				}

				whisper.archives = append(whisper.archives, &narchive)
			}
		} else {
			whisper.archives = append(whisper.archives, archive)
		}
	}

	offset := whisper.MetadataSize()
	for i, retention := range retentions {
		if !whisper.compressed {
			archive := whisper.archives[i]
			archive.offset = offset
			offset += retention.Size()

			continue
		}

		if whisper.aggregationMethod != Mix || i == 0 {
			archive := whisper.archives[i]
			if math.IsNaN(float64(archive.avgCompressedPointSize)) || archive.avgCompressedPointSize <= 0 {
				archive.avgCompressedPointSize = avgCompressedPointSize
			}
			if archive.avgCompressedPointSize > MaxCompressedPointSize {
				archive.avgCompressedPointSize = MaxCompressedPointSize
			}

			archive.cblock.lastByteBitPos = 7
			ppb := archive.calculateSuitablePointsPerBlock(whisper.pointsPerBlock)
			archive.blockSize = int(math.Ceil(float64(ppb)*float64(archive.avgCompressedPointSize))) + endOfBlockSize
			archive.blockRanges = make([]blockRange, archive.blockCount)

			archive.offset = offset
			offset += archive.blockSize * archive.blockCount

			if i > 0 {
				size := archive.secondsPerPoint / whisper.archives[i-1].secondsPerPoint * PointSize * 2
				whisper.archives[i-1].buffer = make([]byte, size)
			}

			continue
		}

		for j := range options.MixAggregationSpecs {
			archive := whisper.archives[1+(i-1)*len(options.MixAggregationSpecs)+j]
			archive.cblock.lastByteBitPos = 7
			archive.blockSize = int(math.Ceil(float64(whisper.pointsPerBlock)*float64(archive.avgCompressedPointSize))) + endOfBlockSize
			archive.blockRanges = make([]blockRange, archive.blockCount)

			archive.offset = offset
			offset += archive.blockSize * archive.blockCount
		}
	}

	if whisper.compressed {
		whisper.initMetaInfo()
		err = whisper.WriteHeaderCompressed()
	} else {
		err = whisper.writeHeader()
	}
	if err != nil {
		return nil, err
	}

	// pre-allocate file size, fallocate proved slower
	//
	// compressed format ignores sparse flag
	if options.Sparse && !options.Compressed {
		if _, err = whisper.file.Seek(int64(whisper.Size()-1), 0); err != nil {
			return nil, err
		}
		if _, err = whisper.file.Write([]byte{0}); err != nil {
			return nil, err
		}
	} else {
		if err := allocateDiskSpace(whisper.file, whisper.Size()-whisper.MetadataSize()); err != nil {
			return nil, err
		}
	}

	return whisper, nil
}

func isGoodFloat32(n float32) bool {
	return !math.IsNaN(float64(n)) && n > 0.0
}

func (whisper *Whisper) blockCount(archive *archiveInfo) int {
	return int(math.Ceil(float64(archive.numberOfPoints)/float64(archive.calculateSuitablePointsPerBlock(whisper.pointsPerBlock)))) + 1
}

func allocateDiskSpace(file file, remaining int) error {
	chunkSize := 16384
	zeros := make([]byte, chunkSize)
	for remaining > chunkSize {
		if _, err := file.Write(zeros); err != nil {
			return err
		}
		remaining -= chunkSize
	}
	if _, err := file.Write(zeros[:remaining]); err != nil {
		return err
	}
	return nil
}

func validateRetentions(retentions Retentions) error {
	if len(retentions) == 0 {
		return fmt.Errorf("no retentions")
	}
	for i, retention := range retentions {
		if i == len(retentions)-1 {
			break
		}

		nextRetention := retentions[i+1]
		if !(retention.secondsPerPoint < nextRetention.secondsPerPoint) {
			return fmt.Errorf("a whisper database may not be configured having two archives with the same precision (archive%v: %v, archive%v: %v)", i, retention, i+1, nextRetention)
		}

		if mod(nextRetention.secondsPerPoint, retention.secondsPerPoint) != 0 {
			return fmt.Errorf("higher precision archives' precision must evenly divide all lower precision archives' precision (archive%v: %v, archive%v: %v)", i, retention.secondsPerPoint, i+1, nextRetention.secondsPerPoint)
		}

		if retention.MaxRetention() >= nextRetention.MaxRetention() {
			return fmt.Errorf("lower precision archives must cover larger time intervals than higher precision archives (archive%v: %v seconds, archive%v: %v seconds)", i, retention.MaxRetention(), i+1, nextRetention.MaxRetention())
		}

		if retention.numberOfPoints < (nextRetention.secondsPerPoint / retention.secondsPerPoint) {
			return fmt.Errorf("each archive must have at least enough points to consolidate to the next archive (archive%v consolidates %v of archive%v's points but it has only %v total points)", i+1, nextRetention.secondsPerPoint/retention.secondsPerPoint, i, retention.numberOfPoints)
		}
	}

	// TODO: cwhisper has more strict retention limit, everything is aggregated from the first archive/retention
	return nil
}

/*
  Open an existing Whisper database and read it's header
*/
func Open(path string) (whisper *Whisper, err error) {
	return OpenWithOptions(path, &Options{
		FLock: false,
	})
}

func OpenWithOptions(path string, options *Options) (whisper *Whisper, err error) {
	var file file
	if options.InMemory {
		file = newMemFile(path)
	} else {
		flag := os.O_RDWR
		if options.OpenFileFlag != nil {
			flag = *options.OpenFileFlag
		}
		file, err = os.OpenFile(path, flag, 0666) // skipcq: GSC-G302
	}
	if err != nil {
		return
	}

	defer func() {
		if err != nil {
			whisper = nil
			file.Close()
		}
	}()

	if options.FLock {
		if err = syscall.Flock(int(file.Fd()), syscall.LOCK_EX); err != nil {
			return
		}
	}

	whisper = new(Whisper)
	whisper.file = file
	whisper.opts = options

	b := make([]byte, len(compressedMagicString))
	if _, err := whisper.file.Read(b); err != nil {
		return nil, fmt.Errorf("unable to read magic string: %s", err)
	} else if bytes.Equal(b, compressedMagicString) {
		whisper.compressed = true
	} else if _, err := whisper.file.Seek(0, 0); err != nil {
		return nil, fmt.Errorf("unable to reset file offset: %s", err)
	}

	// read the metadata
	if whisper.compressed {
		return whisper, whisper.readHeaderCompressed()
	}

	b = make([]byte, MetadataSize)
	readed, err := file.Read(b)
	offset := 0

	if err != nil {
		err = fmt.Errorf("unable to read header: %s", err.Error())
		return
	}
	if readed != MetadataSize {
		err = fmt.Errorf("unable to read header: EOF")
		return
	}

	a := unpackInt(b[offset : offset+IntSize])
	if a > 1024 { // support very old format. File starts with lastUpdate and has only average aggregation method
		whisper.aggregationMethod = Average
	} else {
		whisper.aggregationMethod = AggregationMethod(a)
	}
	offset += IntSize
	whisper.maxRetention = unpackInt(b[offset : offset+IntSize])
	offset += IntSize
	whisper.xFilesFactor = unpackFloat32(b[offset : offset+FloatSize])
	offset += FloatSize
	archiveCount := unpackInt(b[offset : offset+IntSize])
	offset += IntSize

	// read the archive info
	b = make([]byte, ArchiveInfoSize)

	whisper.archives = make([]*archiveInfo, 0)
	for i := 0; i < archiveCount; i++ {
		readed, err = file.Read(b)
		if err != nil || readed != ArchiveInfoSize {
			err = fmt.Errorf("unable to read archive %d metadata: %s", i, err)
			return
		}
		whisper.archives = append(whisper.archives, unpackArchiveInfo(b))
	}

	return whisper, nil
}

func (whisper *Whisper) initMetaInfo() {
	for i, arc := range whisper.archives {
		if arc.cblock.lastByteOffset == 0 {
			arc.cblock.lastByteOffset = arc.blockOffset(arc.cblock.index)
		}
		arc.whisper = whisper

		for i := range arc.blockRanges {
			arc.blockRanges[i].index = i
		}

		if i == 0 {
			continue
		}

		prevArc := whisper.archives[i-1]
		prevArc.next = arc

		if whisper.aggregationMethod != Mix && whisper.compVersion == 1 {
			prevArc.bufferSize = arc.secondsPerPoint / prevArc.secondsPerPoint * PointSize * bufferCount
		}
	}
}

func (whisper *Whisper) writeHeader() (err error) {
	b := make([]byte, whisper.MetadataSize())
	i := 0
	i += packInt(b, int(whisper.aggregationMethod), i)
	i += packInt(b, whisper.maxRetention, i)
	i += packFloat32(b, whisper.xFilesFactor, i)
	i += packInt(b, len(whisper.archives), i)
	for _, archive := range whisper.archives {
		i += packInt(b, archive.offset, i)
		i += packInt(b, archive.secondsPerPoint, i)
		i += packInt(b, archive.numberOfPoints, i)
	}
	_, err = whisper.file.Write(b)

	return err
}

// skipcq: SCC-ST1006, RVV-B0013
func (whisper *Whisper) crc32Offset() int {
	const crc32Size = IntSize
	return len(compressedMagicString) + VersionSize + CompressedMetadataSize - crc32Size - FreeCompressedMetadataSize
}

/*
  Close the whisper file
*/
func (whisper *Whisper) Close() error {
	return whisper.file.Close()
}

/*
  Calculate the total number of bytes the Whisper file should be according to the metadata.
*/
func (whisper *Whisper) Size() int {
	size := whisper.MetadataSize()
	for _, archive := range whisper.archives {
		if whisper.compressed {
			size += archive.blockSize * archive.blockCount
		} else {
			size += archive.Size()
		}
	}
	return size
}

/*
  Calculate the number of bytes the metadata section will be.
*/
func (whisper *Whisper) MetadataSize() int {
	if whisper.compressed {
		return len(compressedMagicString) + VersionSize + CompressedMetadataSize + (CompressedArchiveInfoSize * len(whisper.archives)) + whisper.blockRangesSize() + whisper.bufferSize()
	}

	return MetadataSize + (ArchiveInfoSize * len(whisper.archives))
}

func (whisper *Whisper) blockRangesSize() int {
	var blockRangesSize int
	for _, arc := range whisper.archives {
		blockRangesSize += BlockRangeSize * arc.blockCount
	}
	return blockRangesSize
}

func (whisper *Whisper) bufferSize() int {
	if whisper.aggregationMethod == Mix {
		return 0
	}

	if len(whisper.archives) == 0 {
		return 0
	}
	var bufSize int
	for i, arc := range whisper.archives[1:] {
		bufSize += arc.secondsPerPoint / whisper.archives[i].secondsPerPoint * PointSize * bufferCount
	}
	return bufSize
}

/* Return raw aggregation method */
func (whisper *Whisper) AggregationMethod() AggregationMethod { return whisper.aggregationMethod }

/* Return max retention in seconds */
func (whisper *Whisper) MaxRetention() int {
	return whisper.maxRetention
}

/* Return xFilesFactor */
func (whisper *Whisper) XFilesFactor() float32 {
	return whisper.xFilesFactor
}

/* Return retentions */
func (whisper *Whisper) Retentions() []Retention {
	ret := make([]Retention, 0, 4)
	for _, archive := range whisper.archives {
		ret = append(ret, archive.Retention)
	}

	return ret
}

/*
  Update a value in the database.

  If the timestamp is in the future or outside of the maximum retention it will
  fail immediately.
*/
func (whisper *Whisper) Update(value float64, timestamp int) (err error) {
	// recover panics and return as error
	defer func() {
		if e := recover(); e != nil {
			err = errors.New(e.(string))
		}
	}()

	diff := int(Now().Unix()) - timestamp
	if !(diff < whisper.maxRetention && diff >= 0) {
		return fmt.Errorf("timestamp not covered by any archives in this database")
	}
	var archive *archiveInfo
	var lowerArchives []*archiveInfo
	var i int
	for i, archive = range whisper.archives {
		if archive.MaxRetention() < diff {
			continue
		}
		lowerArchives = whisper.archives[i+1:] // TODO: investigate just returning the positions
		break
	}

	myInterval := timestamp - mod(timestamp, archive.secondsPerPoint)
	point := dataPoint{myInterval, value}

	_, err = whisper.file.WriteAt(point.Bytes(), whisper.getPointOffset(myInterval, archive))
	if err != nil {
		return err
	}

	higher := archive
	for _, lower := range lowerArchives {
		propagated, err := whisper.propagate(myInterval, higher, lower)
		if err != nil {
			return err
		} else if !propagated {
			break
		}
		higher = lower
	}

	return nil
}

func reversePoints(points []*TimeSeriesPoint) {
	size := len(points)
	end := size / 2

	for i := 0; i < end; i++ {
		points[i], points[size-i-1] = points[size-i-1], points[i]
	}
}

var Now = time.Now

func (whisper *Whisper) UpdateMany(points []*TimeSeriesPoint) (err error) {
	return whisper.UpdateManyForArchive(points, -1)
}

/*
  Returns updated amount of out-of-order discarded points since opening whisper file
*/
func (whisper *Whisper) GetDiscardedPointsSinceOpen() uint32 {
	var discardedPointsNow uint32
	if whisper.compressed {
		for i := 0; i < len(whisper.archives); i++ {
			archive := whisper.archives[i]
			if archive.stats.discard.oldInterval > 0 {
				discardedPointsNow += archive.stats.discard.oldInterval
			}
		}
	}
	if discardedPointsNow > whisper.discardedPointsAtOpen {
		return discardedPointsNow - whisper.discardedPointsAtOpen
	}
	return 0
}

// Note: for compressed format, extensions is triggered after update is
// done, so updates of the same data set being done in one
// UpdateManyForArchive call would have different result in file than in
// many UpdateManyForArchive calls.
func (whisper *Whisper) UpdateManyForArchive(points []*TimeSeriesPoint, targetRetention int) (err error) {
	// recover panics and return as error
	defer func() {
		if e := recover(); e != nil {
			err = fmt.Errorf("update_many_for_archive panics: %s\n%s", e, debug.Stack())
		}
	}()

	// sort the points, newest first
	reversePoints(points)
	sort.Stable(timeSeriesPointsNewestFirst{points})

	now := int(Now().Unix()) // TODO: danger of 2030 something overflow

	var currentPoints []*TimeSeriesPoint
	for i := 0; i < len(whisper.archives); i++ {
		archive := whisper.archives[i]

		if targetRetention != -1 && targetRetention != archive.MaxRetention() {
			continue
		}

		if whisper.opts.IgnoreNowOnWrite {
			currentPoints = points
			points = []*TimeSeriesPoint{}
		} else {
			currentPoints, points = extractPoints(points, now, archive.MaxRetention())
		}

		if len(currentPoints) == 0 {
			continue
		}

		// reverse currentPoints
		reversePoints(currentPoints)
		if whisper.compressed {
			// Backfilling lower archives is not allowed/supported for mix
			// aggreation policy with the current api, a new api/parameter is
			// needed to specify which aggregaton target to backfill.
			if whisper.aggregationMethod == Mix && i > 0 {
				break
			}

			// TODO: add a new options to update data points in smaller chunks if
			// it exceeeds certain size, so extension could be triggered
			// properly: ChunkUpdateSize
			err = whisper.archiveUpdateManyCompressed(archive, currentPoints)
		} else {
			err = whisper.archiveUpdateMany(archive, currentPoints)
		}
		if err != nil {
			return
		}
		if len(points) == 0 { // nothing left to do
			break
		}
	}

	if whisper.compressed {
		if err := whisper.WriteHeaderCompressed(); err != nil {
			return err
		}

		if err := whisper.extendIfNeeded(); err != nil {
			return err
		}
	}

	return
}

func (whisper *Whisper) archiveUpdateMany(archive *archiveInfo, points []*TimeSeriesPoint) error {
	alignedPoints := alignPoints(archive, points)
	return whisper.archiveUpdateManyDataPoints(archive, alignedPoints, true)
}

// skipcq: RVV-A0005
func (whisper *Whisper) archiveUpdateManyDataPoints(archive *archiveInfo, alignedPoints []dataPoint, propagate bool) error {
	intervals, packedBlocks := packSequences(archive, alignedPoints)

	baseInterval := whisper.getBaseInterval(archive)
	if baseInterval == 0 {
		baseInterval = intervals[0]
	}

	for i := range intervals {
		myOffset := archive.PointOffset(baseInterval, intervals[i])
		bytesBeyond := int(myOffset-archive.End()) + len(packedBlocks[i])
		if bytesBeyond > 0 {
			pos := len(packedBlocks[i]) - bytesBeyond
			err := whisper.fileWriteAt(packedBlocks[i][:pos], myOffset)
			if err != nil {
				return err
			}
			err = whisper.fileWriteAt(packedBlocks[i][pos:], archive.Offset())
			if err != nil {
				return err
			}
		} else {
			err := whisper.fileWriteAt(packedBlocks[i], myOffset)
			if err != nil {
				return err
			}
		}
	}

	if !propagate {
		return nil
	}

	higher := archive
	lowerArchives := whisper.lowerArchives(archive)

	for _, lower := range lowerArchives {
		seen := make(map[int]bool)
		propagateFurther := false
		for _, point := range alignedPoints {
			interval := point.interval - mod(point.interval, lower.secondsPerPoint)
			if !seen[interval] {
				if propagated, err := whisper.propagate(interval, higher, lower); err != nil {
					panic("Failed to propagate")
				} else if propagated {
					propagateFurther = true
				}
			}
		}
		if !propagateFurther {
			break
		}
		higher = lower
	}

	return nil
}

func extractPoints(points []*TimeSeriesPoint, now, maxRetention int) (currentPoints []*TimeSeriesPoint, remainingPoints []*TimeSeriesPoint) {
	maxAge := now - maxRetention
	for i, point := range points {
		if point.Time < maxAge {
			if i > 0 {
				return points[:i-1], points[i-1:]
			} else {
				return []*TimeSeriesPoint{}, points
			}
		}
	}
	return points, remainingPoints
}

func alignPoints(archive *archiveInfo, points []*TimeSeriesPoint) []dataPoint {
	alignedPoints := make([]dataPoint, 0, len(points))
	positions := make(map[int]int)
	for _, point := range points {
		dPoint := dataPoint{point.Time - mod(point.Time, archive.secondsPerPoint), point.Value}
		if p, ok := positions[dPoint.interval]; ok {
			alignedPoints[p] = dPoint
		} else {
			alignedPoints = append(alignedPoints, dPoint)
			positions[dPoint.interval] = len(alignedPoints) - 1
		}
	}
	return alignedPoints
}

func packSequences(archive *archiveInfo, points []dataPoint) (intervals []int, packedBlocks [][]byte) {
	intervals = make([]int, 0)
	packedBlocks = make([][]byte, 0)
	var prevInterval int
	for i, point := range points {
		if i == 0 || point.interval != prevInterval+archive.secondsPerPoint {
			intervals = append(intervals, point.interval)
			packedBlocks = append(packedBlocks, point.Bytes())
		} else {
			packedBlocks[len(packedBlocks)-1] = append(packedBlocks[len(packedBlocks)-1], point.Bytes()...)
		}
		prevInterval = point.interval
	}
	return
}

/*
	Calculate the offset for a given interval in an archive

	This method retrieves the baseInterval and the
*/
func (whisper *Whisper) getPointOffset(start int, archive *archiveInfo) int64 {
	baseInterval := whisper.getBaseInterval(archive)
	if baseInterval == 0 {
		return archive.Offset()
	}
	return archive.PointOffset(baseInterval, start)
}

func (whisper *Whisper) getBaseInterval(archive *archiveInfo) int {
	if whisper.compressed {
		return unpackInt(archive.buffer)
	}

	baseInterval, err := whisper.readInt(archive.Offset())
	if err != nil {
		panic(fmt.Sprintf("Failed to read baseInterval: %s", err.Error()))
	}
	return baseInterval
}

func (whisper *Whisper) lowerArchives(archive *archiveInfo) (lowerArchives []*archiveInfo) {
	for i, lower := range whisper.archives {
		if lower.secondsPerPoint > archive.secondsPerPoint {
			return whisper.archives[i:]
		}
	}
	return
}

func (whisper *Whisper) propagate(timestamp int, higher, lower *archiveInfo) (bool, error) {
	lowerIntervalStart := timestamp - mod(timestamp, lower.secondsPerPoint)

	higherFirstOffset := whisper.getPointOffset(lowerIntervalStart, higher)

	// TODO: extract all this series extraction stuff
	higherPoints := lower.secondsPerPoint / higher.secondsPerPoint
	higherSize := higherPoints * PointSize
	relativeFirstOffset := higherFirstOffset - higher.Offset()
	relativeLastOffset := int64(mod(int(relativeFirstOffset+int64(higherSize)), higher.Size()))
	higherLastOffset := relativeLastOffset + higher.Offset()

	series, err := whisper.readSeries(higherFirstOffset, higherLastOffset, higher)
	if err != nil {
		return false, err
	}

	// and finally we construct a list of values
	knownValues := make([]float64, 0, len(series))
	currentInterval := lowerIntervalStart

	for _, dPoint := range series {
		if dPoint.interval == currentInterval {
			knownValues = append(knownValues, dPoint.value)
		}
		currentInterval += higher.secondsPerPoint
	}

	// propagate aggregateValue to propagate from neighborValues if we have enough known points
	if len(knownValues) == 0 {
		return false, nil
	}

	knownPercent := float32(len(knownValues)) / float32(len(series))
	if knownPercent < whisper.xFilesFactor { // check we have enough data points to propagate a value
		return false, nil
	} else {
		aggregateValue := aggregate(whisper.aggregationMethod, knownValues)
		point := dataPoint{lowerIntervalStart, aggregateValue}
		if _, err := whisper.file.WriteAt(point.Bytes(), whisper.getPointOffset(lowerIntervalStart, lower)); err != nil {
			return false, err
		}
	}
	return true, nil
}

func (whisper *Whisper) readSeries(start, end int64, archive *archiveInfo) ([]dataPoint, error) {
	var b []byte
	if start < end {
		b = make([]byte, end-start)
		err := whisper.fileReadAt(b, start)
		if err != nil {
			return nil, err
		}
	} else {
		b = make([]byte, archive.End()-start)
		err := whisper.fileReadAt(b, start)
		if err != nil {
			return nil, err
		}
		b2 := make([]byte, end-archive.Offset())
		err = whisper.fileReadAt(b2, archive.Offset())
		if err != nil {
			return nil, err
		}
		b = append(b, b2...)
	}
	return unpackDataPoints(b), nil
}

func (whisper *Whisper) checkSeriesEmpty(start, end int64, archive *archiveInfo, fromTime, untilTime int) (bool, error) {
	if start < end {
		length := end - start
		return whisper.checkSeriesEmptyAt(start, length, fromTime, untilTime)
	}
	length := archive.End() - start
	empty, err := whisper.checkSeriesEmptyAt(start, length, fromTime, untilTime)
	if err != nil || !empty {
		return empty, err
	}
	return whisper.checkSeriesEmptyAt(archive.Offset(), end-archive.Offset(), fromTime, untilTime)

}

func (whisper *Whisper) checkSeriesEmptyAt(start, length int64, fromTime, untilTime int) (bool, error) {
	b1 := make([]byte, 4)
	// Read first point
	err := whisper.fileReadAt(b1, start)
	if err != nil {
		return false, err
	}
	pointTime := unpackInt(b1)
	if pointTime > fromTime && pointTime < untilTime {
		return false, nil
	}

	b2 := make([]byte, 4)
	// Read last point
	err = whisper.fileReadAt(b2, length-12)
	if err != nil {
		return false, err
	}
	pointTime = unpackInt(b1)
	if pointTime > fromTime && pointTime < untilTime {
		return false, nil
	}
	return true, nil
}

/*
  Calculate the starting time for a whisper db.
*/
func (whisper *Whisper) StartTime() int {
	now := int(Now().Unix()) // TODO: danger of 2030 something overflow
	return now - whisper.maxRetention
}

/*
  Fetch a TimeSeries for a given time span from the file.
*/
func (whisper *Whisper) Fetch(fromTime, untilTime int) (timeSeries *TimeSeries, err error) {
	return whisper.FetchByAggregation(fromTime, untilTime, nil)
}

func (whisper *Whisper) FetchByAggregation(fromTime, untilTime int, spec *MixAggregationSpec) (timeSeries *TimeSeries, err error) {
	now := int(Now().Unix()) // TODO: danger of 2030 something overflow
	if fromTime > untilTime {
		return nil, fmt.Errorf("invalid time interval: from time '%d' is after until time '%d'", fromTime, untilTime)
	}
	// oldestTime := whisper.StartTime()
	oldestTime := now - whisper.maxRetention
	// range is in the future
	if fromTime > now {
		return nil, nil
	}
	// range is beyond retention
	if untilTime < oldestTime {
		return nil, nil
	}
	if fromTime < oldestTime {
		fromTime = oldestTime
	}
	if untilTime > now {
		untilTime = now
	}

	// TODO: improve this algorithm it's ugly
	diff := now - fromTime
	var archive *archiveInfo
	for _, archive = range whisper.archives {
		if archive.MaxRetention() >= diff {
			// TODO: select a default aggregation policy?
			if whisper.aggregationMethod == Mix && spec != nil && archive.aggregationSpec != nil && spec.String() != archive.aggregationSpec.String() {
				continue
			}

			break
		}
	}
	// NOTE: base (i.e. the first) archive should have a nil aggregationSpec
	if whisper.aggregationMethod == Mix && spec != nil && archive.aggregationSpec != nil && spec.String() != archive.aggregationSpec.String() {
		return nil, fmt.Errorf("target aggregation %s not found", spec)
	}

	return whisper.fetchFromArchive(archive, fromTime, untilTime)
}

func (whisper *Whisper) fetchFromArchive(archive *archiveInfo, fromTime, untilTime int) (timeSeries *TimeSeries, err error) {
	fromInterval := archive.Interval(fromTime)
	untilInterval := archive.Interval(untilTime)

	var series []dataPoint
	if whisper.compressed {
		series, err = whisper.fetchCompressed(int64(fromInterval), int64(untilInterval), archive)
		if err != nil {
			return nil, err
		}

		irange := untilInterval - fromInterval
		values := make([]float64, irange/archive.secondsPerPoint)

		for i := range values {
			values[i] = math.NaN()
		}
		step := archive.secondsPerPoint
		for _, dPoint := range series {
			index := (dPoint.interval - fromInterval) / archive.secondsPerPoint
			if index >= len(values) {
				break
			}
			values[index] = dPoint.value
		}
		return &TimeSeries{fromInterval, untilInterval, step, values}, nil
	} else {
		baseInterval := whisper.getBaseInterval(archive)

		if baseInterval == 0 {
			step := archive.secondsPerPoint
			points := (untilInterval - fromInterval) / step
			values := make([]float64, points)
			for i := range values {
				values[i] = math.NaN()
			}
			return &TimeSeries{fromInterval, untilInterval, step, values}, nil
		}

		// Zero-length time range: always include the next point
		if fromInterval == untilInterval {
			untilInterval += archive.SecondsPerPoint()
		}

		fromOffset := archive.PointOffset(baseInterval, fromInterval)
		untilOffset := archive.PointOffset(baseInterval, untilInterval)

		series, err = whisper.readSeries(fromOffset, untilOffset, archive)
		if err != nil {
			return nil, err
		}

		values := make([]float64, len(series))
		for i := range values {
			values[i] = math.NaN()
		}
		currentInterval := fromInterval
		step := archive.secondsPerPoint

		for i, dPoint := range series {
			if dPoint.interval == currentInterval {
				values[i] = dPoint.value
			}
			currentInterval += step
		}
		return &TimeSeries{fromInterval, untilInterval, step, values}, nil
	}
}

/*
  Check a TimeSeries has a points for a given time span from the file.
*/
func (whisper *Whisper) CheckEmpty(fromTime, untilTime int) (exist bool, err error) {
	now := int(Now().Unix()) // TODO: danger of 2030 something overflow
	if fromTime > untilTime {
		return true, fmt.Errorf("invalid time interval: from time '%d' is after until time '%d'", fromTime, untilTime)
	}
	oldestTime := whisper.StartTime()
	// range is in the future
	if fromTime > now {
		return true, nil
	}
	// range is beyond retention
	if untilTime < oldestTime {
		return true, nil
	}
	if fromTime < oldestTime {
		fromTime = oldestTime
	}
	if untilTime > now {
		untilTime = now
	}

	// TODO: improve this algorithm it's ugly
	diff := now - fromTime
	var archive *archiveInfo
	for _, archive = range whisper.archives {
		fromInterval := archive.Interval(fromTime)
		untilInterval := archive.Interval(untilTime)
		baseInterval := whisper.getBaseInterval(archive)

		if baseInterval == 0 {
			return true, nil
		}

		// Zero-length time range: always include the next point
		if fromInterval == untilInterval {
			untilInterval += archive.SecondsPerPoint()
		}

		fromOffset := archive.PointOffset(baseInterval, fromInterval)
		untilOffset := archive.PointOffset(baseInterval, untilInterval)

		empty, err := whisper.checkSeriesEmpty(fromOffset, untilOffset, archive, fromTime, untilTime)
		if err != nil || !empty {
			return empty, err
		}
		if archive.MaxRetention() >= diff {
			break
		}
	}
	return true, nil
}

func (whisper *Whisper) readInt(offset int64) (int, error) {
	// TODO: make errors better
	b := make([]byte, IntSize)
	_, err := whisper.file.ReadAt(b, offset)
	if err != nil {
		return 0, err
	}

	return unpackInt(b), nil
}

func (r *Retention) MaxRetention() int                      { return r.secondsPerPoint * r.numberOfPoints }
func (r *Retention) Size() int                              { return r.numberOfPoints * PointSize }
func (r *Retention) SecondsPerPoint() int                   { return r.secondsPerPoint }
func (r *Retention) NumberOfPoints() int                    { return r.numberOfPoints }
func (r *Retention) SetAvgCompressedPointSize(size float32) { r.avgCompressedPointSize = size }

// NOTE: the calculation result is not saved on disk
func (r *Retention) calculateSuitablePointsPerBlock(defaultSize int) int {
	if defaultSize == 0 {
		defaultSize = DefaultPointsPerBlock
	}

	if r.numberOfPoints >= defaultSize*3 {
		return defaultSize
	} else if r.numberOfPoints > 256 {
		return (r.numberOfPoints + 16) / 4
	}
	return r.numberOfPoints + 16
}

func durationString(v int) string {
	switch {
	case v >= 365*24*60*60:
		return fmt.Sprintf("%dy", v/(365*24*60*60))
	case v >= 24*60*60:
		return fmt.Sprintf("%dd", v/(24*60*60))
	case v >= 60*60:
		return fmt.Sprintf("%dh", v/(60*60))
	case v >= 60:
		return fmt.Sprintf("%dm", v/(60))
	default:
		return fmt.Sprintf("%ds", v)
	}
}

func (r Retention) String() string {
	return fmt.Sprintf("%s:%s", durationString(r.secondsPerPoint), durationString(r.secondsPerPoint*r.numberOfPoints))
}

func NewRetention(secondsPerPoint, numberOfPoints int) Retention {
	return Retention{
		secondsPerPoint: secondsPerPoint,
		numberOfPoints:  numberOfPoints,
	}
}

// TODO: maybe we should make it array of structs, rather than an array of struct pointers.
type Retentions []*Retention

func (r Retentions) Len() int {
	return len(r)
}

func (r Retentions) Swap(i, j int) {
	r[i], r[j] = r[j], r[i]
}

func NewRetentionsNoPointer(r2 []Retention) Retentions {
	var nr2 Retentions
	for i := range r2 {
		nr2 = append(nr2, &r2[i])
	}

	return nr2
}

func (r1 Retentions) Equal(r2 Retentions) bool {
	if len(r1) != len(r2) {
		return false
	}
	for i := range r1 {
		if r1[i].secondsPerPoint != r2[i].secondsPerPoint {
			return false
		}
		if r1[i].numberOfPoints != r2[i].numberOfPoints {
			return false
		}
	}

	return true
}

type retentionsByPrecision struct{ Retentions }

func (r retentionsByPrecision) Less(i, j int) bool {
	return r.Retentions[i].secondsPerPoint < r.Retentions[j].secondsPerPoint
}

func (archive *archiveInfo) Offset() int64 {
	return int64(archive.offset)
}

func (archive *archiveInfo) PointOffset(baseInterval, interval int) int64 {
	timeDistance := interval - baseInterval
	pointDistance := timeDistance / archive.secondsPerPoint
	byteDistance := pointDistance * PointSize
	myOffset := archive.Offset() + int64(mod(byteDistance, archive.Size()))

	return myOffset
}

func (archive *archiveInfo) End() int64 {
	return archive.Offset() + int64(archive.Size())
}

func (archive *archiveInfo) Interval(time int) int {
	return time - mod(time, archive.secondsPerPoint) + archive.secondsPerPoint
}

func (archive *archiveInfo) AggregateInterval(time int) int {
	return time - mod(time, archive.next.secondsPerPoint)
}

type TimeSeries struct {
	fromTime  int
	untilTime int
	step      int
	values    []float64
}

func (ts *TimeSeries) FromTime() int {
	return ts.fromTime
}

func (ts *TimeSeries) UntilTime() int {
	return ts.untilTime
}

func (ts *TimeSeries) Step() int {
	return ts.step
}

func (ts *TimeSeries) Values() []float64 {
	return ts.values
}

func (ts *TimeSeries) Points() []TimeSeriesPoint {
	points := make([]TimeSeriesPoint, len(ts.values))
	for i, value := range ts.values {
		points[i] = TimeSeriesPoint{Time: ts.fromTime + ts.step*i, Value: value}
	}
	return points
}

func (ts *TimeSeries) PointPointers() []*TimeSeriesPoint {
	points := make([]*TimeSeriesPoint, len(ts.values))
	for i, value := range ts.values {
		points[i] = &TimeSeriesPoint{Time: ts.fromTime + ts.step*i, Value: value}
	}
	return points
}

func (ts *TimeSeries) String() string {
	return fmt.Sprintf("TimeSeries{'%v' '%-v' %v %v}", time.Unix(int64(ts.fromTime), 0), time.Unix(int64(ts.untilTime), 0), ts.step, ts.values)
}

type TimeSeriesPoint struct {
	Time  int
	Value float64
}

type timeSeriesPoints []*TimeSeriesPoint

func (p timeSeriesPoints) Len() int {
	return len(p)
}

func (p timeSeriesPoints) Swap(i, j int) {
	p[i], p[j] = p[j], p[i]
}

type timeSeriesPointsNewestFirst struct {
	timeSeriesPoints
}

func (p timeSeriesPointsNewestFirst) Less(i, j int) bool {
	return p.timeSeriesPoints[i].Time > p.timeSeriesPoints[j].Time
}

type dataPoint struct {
	interval int
	value    float64
}

func (point *dataPoint) Bytes() []byte {
	b := make([]byte, PointSize)
	packInt(b, point.interval, 0)
	packFloat64(b, point.value, IntSize)
	return b
}

func sum(values []float64) float64 {
	result := 0.0
	for _, value := range values {
		result += value
	}
	return result
}

func aggregate(method AggregationMethod, knownValues []float64) float64 {
	switch method {
	case Average:
		return sum(knownValues) / float64(len(knownValues))
	case Sum:
		return sum(knownValues)
	case First:
		return knownValues[0]
	case Last:
		return knownValues[len(knownValues)-1]
	case Max:
		max := knownValues[0]
		for _, value := range knownValues {
			if value > max {
				max = value
			}
		}
		return max
	case Min:
		min := knownValues[0]
		for _, value := range knownValues {
			if value < min {
				min = value
			}
		}
		return min
	}
	panic(fmt.Sprintf("Invalid aggregation method: %d", method))
}

func packInt(b []byte, v, i int) int {
	binary.BigEndian.PutUint32(b[i:i+IntSize], uint32(v))
	return IntSize
}

func packFloat32(b []byte, v float32, i int) int {
	binary.BigEndian.PutUint32(b[i:i+FloatSize], math.Float32bits(v))
	return FloatSize
}

func packFloat64(b []byte, v float64, i int) int {
	binary.BigEndian.PutUint64(b[i:i+Float64Size], math.Float64bits(v))
	return Float64Size
}

func unpackInt(b []byte) int {
	return int(binary.BigEndian.Uint32(b))
}

func unpackFloat32(b []byte) float32 {
	return math.Float32frombits(binary.BigEndian.Uint32(b))
}

func unpackFloat64(b []byte) float64 {
	return math.Float64frombits(binary.BigEndian.Uint64(b))
}

func unpackArchiveInfo(b []byte) *archiveInfo {
	return &archiveInfo{
		Retention: Retention{secondsPerPoint: unpackInt(b[IntSize : IntSize*2]), numberOfPoints: unpackInt(b[IntSize*2 : IntSize*3])},
		offset:    unpackInt(b[:IntSize]),
	}
}

func unpackDataPoint(b []byte) dataPoint {
	return dataPoint{unpackInt(b[0:IntSize]), unpackFloat64(b[IntSize:PointSize])}
}

func unpackDataPoints(b []byte) (series []dataPoint) {
	series = make([]dataPoint, 0, len(b)/PointSize)
	for i := 0; i < len(b); i += PointSize {
		series = append(series, unpackDataPoint(b[i:i+PointSize]))
	}
	return
}

func unpackDataPointsStrict(b []byte) (series []dataPoint) {
	series = make([]dataPoint, 0, len(b)/PointSize)
	for i := 0; i < len(b); i += PointSize {
		dp := unpackDataPoint(b[i : i+PointSize])
		if dp.interval == 0 {
			continue
		}
		series = append(series, dp)
	}
	return
}

func getFirstDataPointStrict(b []byte) dataPoint {
	for i := 0; i < len(b); i += PointSize {
		dp := unpackDataPoint(b[i : i+PointSize])
		if dp.interval == 0 {
			continue
		}
		return dp
	}
	return dataPoint{}
}

/*
	Implementation of modulo that works like Python
	Thanks @timmow for this
*/
func mod(a, b int) int {
	return a - (b * int(math.Floor(float64(a)/float64(b))))
}

// TODO: optmize with assembly
// from https://create.stephan-brumme.com/crc32/
func crc32(data []byte, prev uint32) uint32 {
	const polynomial uint32 = 0xEDB88320

	crc := prev ^ 0xFFFFFFFF
	for _, b := range data {
		crc ^= uint32(b)
		for i := 0; i < 8; i++ {
			crc = (crc >> 1) ^ (uint32(-1*int32(crc&1)) & polynomial)
		}
	}
	return crc ^ 0xFFFFFFFF
}

func (whisper *Whisper) File() *os.File { return whisper.file.(*os.File) }

func (mas *MixAggregationSpec) String() string {
	if mas.Method == Percentile {
		if float32(int(mas.Percentile)) == mas.Percentile {
			return fmt.Sprintf("p%0.0f", mas.Percentile)
		}
		return fmt.Sprintf("p%f", mas.Percentile)
	}
	return mas.Method.String()
}

func (whisper *Whisper) HasMatchingConfigs(rets Retentions, aggr AggregationMethod, xff float32, options *Options) bool {
	if !rets.Equal(NewRetentionsNoPointer(whisper.Retentions())) {
		return false
	}
	if whisper.aggregationMethod != aggr {
		return false
	}
	if whisper.xFilesFactor != xff {
		return false
	}

	// TODO: support whisper -> cwhisper conversion and vice versa
	// skipcq: SCC-SA9003
	if whisper.compressed != options.Compressed {
		// 	return false
	}

	return true
}

func (whisper *Whisper) UpdateConfig(rets Retentions, aggr AggregationMethod, xff float32, options *Options) (err error) {
	newFilename := whisper.file.Name() + ".migrate"
	os.Remove(newFilename) // in case there are broken/corrupted migrate files not being cleaned up properly

	defer func() {
		if r := recover(); r != nil {
			// make sure that temporary file are removed properly
			os.Remove(newFilename)

			err = fmt.Errorf("update_config panics: %s\n%s", r, debug.Stack())
		}
	}()

	updateRets := !rets.Equal(NewRetentionsNoPointer(whisper.Retentions()))
	updateAggrXff := whisper.aggregationMethod != aggr || whisper.xFilesFactor != xff

	if updateRets {
		newWhisper, err := CreateWithOptions(newFilename, rets, aggr, xff, options)
		if err != nil {
			return err
		}

		if options.Compressed {
			err = newWhisper.FillCompressed(whisper)
		} else {
			err = newWhisper.FillClassic(whisper)
		}

		// error ignored here, not much we can do if we fail to close files?
		newWhisper.Close()
		whisper.Close()

		if err != nil {
			return err
		}

		// how about aggregations: go-carbon will use the new aggr when updating retentions
		return os.Rename(newFilename, whisper.file.Name())
	}

	if updateAggrXff {
		// no need to create a new whisper file, just do in-place update
		// for aggregations and xff. histories won't be changed.
		whisper.xFilesFactor = xff
		whisper.aggregationMethod = aggr

		aggData := make([]byte, IntSize)
		packInt(aggData, int(whisper.aggregationMethod), 0)
		xffData := make([]byte, FloatSize)
		packFloat32(xffData, whisper.xFilesFactor, 0)

		if options.Compressed {
			if _, err := whisper.file.WriteAt(aggData, int64(compressedHeaderAggregationOffset)); err != nil {
				return err
			}
			if _, err := whisper.file.WriteAt(xffData, int64(compressedHeaderXFFOffset)); err != nil {
				return err
			}
		} else {
			if _, err := whisper.file.WriteAt(aggData, int64(classicHeaderAggregationOffset)); err != nil {
				return err
			}
			if _, err := whisper.file.WriteAt(xffData, int64(classicHeaderXFFOffset)); err != nil {
				return err
			}
		}

		if _, err := whisper.file.(*os.File).Seek(0, 0); err != nil {
			return err
		}

		return nil
	}

	return nil
}

func (dstw *Whisper) FillClassic(srcw *Whisper) error {
	pointsByArchives, err := dstw.retrieveAndMerge(srcw)
	if err != nil {
		return err
	}

	for i, points := range pointsByArchives {
		if len(points) == 0 {
			continue
		}

		archive := dstw.archives[i]
		if err := dstw.archiveUpdateManyDataPoints(archive, points, false); err != nil {
			return err
		}
	}

	return nil
}

// retrieveAndMerge returns data points from both dst and src whisper files. It tries to
// figure out if src has data points of the same resolution/secondsPerPoint and performs
// merge if the dst has no null datapoints at the same timestamp.
func (dstw *Whisper) retrieveAndMerge(srcw *Whisper) (pointsByArchives [][]dataPoint, err error) {
	defer func() {
		if r := recover(); r != nil {
			err = fmt.Errorf("retrieve_and_merge panics: %s\n%s", r, debug.Stack())
		}
	}()

	// notable cases:
	//   case:
	//     src: 1m:30d,1h:10y
	//     dst: 1m:60d,1h:20y
	//   case:
	//     src: 1m:30d,1h:10y
	//     dst: 1m:60d,1h:5y,1d:100y
	//   case:
	//     src: 1m:30d,1h:10y
	//     dst: 1m:60d,30m:1y,1h:10y
	//   case:
	//     src: 1m:30d,1h:10y
	//     dst: 30s:30d,30m:1y,1h:10y
	//   case:
	//     src: 1m:30d,1h:10y
	//     dst: 1s:4d,1m:60d,1h:20y
	//   case:
	//     src: 1m:30d,1h:10y
	//     dst: 1s:4d,1m:20d,1h:20y

	pointsByArchives = make([][]dataPoint, len(dstw.archives))
	for i, dstArc := range dstw.archives {
		// TODO: fix needed. should limit from/until to the smaller retentions, in src or in dst.
		until := int(Now().Unix())
		from := until - dstArc.MaxRetention()

		var srcArc *archiveInfo
		srcPoints := &TimeSeries{}
		for _, srcArcx := range srcw.archives {
			if srcArcx.secondsPerPoint != dstArc.secondsPerPoint {
				continue
			}

			srcFrom := from
			srcUtil := until
			if dstArc.MaxRetention() > srcArcx.MaxRetention() {
				srcFrom = until - srcArcx.MaxRetention()
			}

			// TODO: more thoughts needed for mix aggregation backfilling?
			if dstw.aggregationMethod == Mix &&
				srcArcx.aggregationSpec != nil &&
				dstArc.aggregationSpec != nil &&
				srcArcx.aggregationSpec.Method != dstArc.aggregationSpec.Method {
				continue
			}

			srcArc = srcArcx
			srcPoints, err = srcw.fetchFromArchive(srcArcx, srcFrom, srcUtil)
			if err != nil {
				return nil, err
			}
		}

		dstPoints, err := dstw.fetchFromArchive(dstArc, from, until)
		if err != nil {
			return nil, err
		}

		if srcArc != nil {
			// in theory, this error shouldn't be happening, but we want to
			// be sure that we fetching and merging values from archives in
			// the same resolution, i.e. secondsPerPoint.
			if srcPoints.step != srcArc.secondsPerPoint || dstPoints.step != dstArc.secondsPerPoint {
				return nil, fmt.Errorf("whisper.retrieveAndMerge: failed to retrieve values from proper archive. src:%s:%s dst:%s:%s", srcArc.Retention, durationString(srcPoints.step), dstArc.Retention, durationString(dstPoints.step))
			}
		}

		var points = make([]dataPoint, len(dstPoints.values))
		var pidx int
		var srcOffset = (dstPoints.fromTime - srcPoints.fromTime) / dstPoints.step

		// step 2
		//   dst:  0 - 60
		//   src: 32 - 60
		// step 1
		//   dst: 32 - 60
		//   src:  0 - 60
		for i, val := range dstPoints.values {
			if !math.IsNaN(val) {
			} else if srcArc != nil &&
				srcArc.secondsPerPoint == dstArc.secondsPerPoint &&
				0 <= i+srcOffset &&
				i+srcOffset < len(srcPoints.values) &&
				!math.IsNaN(srcPoints.values[i+srcOffset]) { // Only copy histories when resolution is the same
				val = srcPoints.values[i+srcOffset]
			} else {
				continue
			}

			points[pidx].interval = dstPoints.fromTime + i*dstArc.secondsPerPoint
			points[pidx].value = val
			pidx++
		}
		points = points[:pidx]

		pointsByArchives[i] = points
	}

	return pointsByArchives, nil
}

redpajama_set_name: RedPajamaGithub


Really loved the text ahead of time with the pic of who would be coming. Cris was prompr, very friendly and addressed all questions, as well as educating us on how the systems run in general.
Man, these guys are great!
Our installers were very friendly and polite. Steve who came to sell us the units was amazing and did all he could to meet our needs. He did not try to over sell us on anything or give us anything our home didn't need.
The furnace still has some issues but your service is so good that I over look the issues at hand.
We got someone here the same day we called and they immediately knew what to do for us. The person who helped us answered all the questions we had, and treated us with respect. I didn't feel like he was pushy at all. Also, when we showed concern about the price, he gave us different options. We really appreciated everything that he did for us.
Wonderful technician. Answered any questions and was very friendly.
Our technician arrived in a timely fashion. He was knowledgeable and helpful and fixed our furnace quickly. It is working now and we will have a warm house this winter, so we are very happy.
tuneup our furnace and replace an important part.
Dennis was great! He was very thorough and knowledgable. He explained the process clearly and was very helpful and willing to answer any and all questions.
Dennis was friendly efficient and knowledgeable. He offered insight and suggestions to help us with our needs.
No criticism for Shay but I was surprised that my AC unit was not top of the line. I purchased an expensive part to reduce start-up duty load.
Representative personable and honest. When you have people come into your home that you don't know, it means a lot to have them be someone you can trust and be able to represent your company honestly.
First off, people were nice. Liked Eric (service guy), but whole experience just wasn't what I expected. I originally had an appointment scheduled two weeks ago (after waiting 2 weeks initially after making first appointment, which was to be expected, ok with that), but then received a call on day of 1st appointment and said that they wouldn't be able to make it due to heavy work load (which I get too). When I asked for rescheduling a new appointment, they said it'd be another 2 weeks before they could get to me. This was the point I began to question the professionalism of your company. I figured since this rescheduling was your fault, you'd work with me a little more and get someone out to my house a little sooner. Not! Said you couldn't manage it. So, the person rescheduling me for 2 weeks later said she give me a $10.00 discount (which never happened in the end, but whatever), and so I rescheduled. Yesterday, Eric showed up, did the "performance check" and said everything looked great except 2 items: ignitor and motor/wheel both bad, since it was vibrating (Eric even grabbed onto the wheel and said the seals wear out and then there's excess movement causing the wheel to be out of balance). He told me I could be confident in his diagnosis since he's been doing this for 15 years, which I didn't question since again, nice guy. I asked him to write up a quote and came back as $1,081.39. I then asked him to send someone over to give me an estimate for a new system since I figured a thousand bucks would be better used to go to that considering the age of my furnace. After meeting with John, nice guy as well, and getting everything in front of me, I decided I had better do some additional homework, which I did. I called a couple of other heating/cooling companies around and spoke to some of their service guys and one of them to instructed me to look further in on the wheel to see if there was anything resting on the wheel (paper, lint ball, etc.), which ultimately could cause the vibration that I was getting (which both Eric and John said were symptoms of a bad motor and wheel). After looking at the wheel further, I did find a rather large piece of insulation on the wheel itself. I simply pulled out the insulation, restarted the furnace and whalaa, smooth running motor and wheel (which cost me $0.00). Fortunately I like the outcome cause I'm a $1,000 ahead, but unfortunately for you, these 2 instances show a serious lack of professionalism. Good luck to you, I'll be looking elsewhere from now on.
Thank you so much for the feedback. We take this type of feedback very seriously so I will make sure our General Manager reaches out to you so we can make things right and continue to improve our service. Hopefully we can earn your business back in the future!
Great service and wonderful agent doing the work.
You were on time and the guy providing the service was very nice and when he finished the service, he asked if I had any other concerns or questions he could help me with.
Nice helpful people. Knowledgeable, good price.


text:
Metal News Post-hardcore
I Prevail Add Show To Tour
September 25, 2017 September 25, 2017 Brooke Daly	Brian Burkheiser, concert, date, Dylan Bowman, Eric Vanlerberghe, escape the fate, fearless records, I Prevail, infectious magazine, live, metal, metalcore, music, new, post hardcore, show, Steve Menoian, the word alive, we came as romans0 comment
I Prevail have one full fall season ahead of them. They're booked through November playing gigs across the country. But, just to ensure they didn't miss anyone, they've added one more date. IP, We Came As Romans, The Word Alive, and Escape The Fate will come to Denver on November 20th. The show was recently announced and was not part of the original routing. Check out the details in a tweet from I Prevail after the jump! For more I Prevail, you can purchase a concert ticket here or buy…
WATCH: The Word Alive Perform New Song Live
September 24, 2017 Brooke Daly	"Red Clouds", escape the fate, fearless records, I Prevail, infectious magazine, live, live footage, matt horn, metal, metalcore, music, new, performance, song, Telle Smith, the word alive, Tony Pizzuti, twa, we came as romans, Zack Hansen0 comment
The Word Alive have been very open about the fact that they're creating a follow-up to their album Dark Matter. In conjunction with some social media teasing, the band are bringing some new sounds to their current tour with I Prevail, We Came As Romans, and Escape The Fate. TWA's set on each date thus far has included a new song, "Red Clouds." A fan (YouTube user xToRm3nTeDz) captured some live footage when the band hit Oklahoma City. So far, the song sounds heavy and catchy while including aspects of their songwriting fans…
We Came As Romans Announce Headlining Shows
September 6, 2017 Brooke Daly	Andy Glass, announcement, Dave Stephens, David Puckett, escape the fate, I Prevail, infectious magazine, Joshua Moore, Kyle Pavone, live, Lou Cotton, metal, metalcore, music, new, performance, SharpTone Records, shows, the word alive, wcar, we came as romans0 comment
When you're touring, you usually use days off to rest, readjust, and get ready for the next show. We Came As Romans, however, are taking their days off and turning them into special occasions. The group announced that they will have several headlining shows throughout the course of their tour with I Prevail. The Word Alive and Escape The Fate will join WCAR as support on their headlining dates. Check out the details in a tweet from WCAR after the jump! For more We Came As Romans, you can purchase…
The Word Alive Begin Work On Fifth Album
August 28, 2017 Brooke Daly	album, announcement, fearless records, follow-up, infectious magazine, Matt Good, matt horn, metal, metalcore, music, recording, Telle Smith, the word alive, Tony Pizzuti, twa, update, Zack Hansen0 comment
The Word Alive are wasting no time creating a follow-up to their 2016 album, Dark Matter. Despite a member change, the group are pushing forward and have begun working on their next album. In a detailed blog post on their Facebook, the band discussed what's happening moving forward. TWA confirmed they're hard at work and that Matt Good will produce the effort. Check out all of the details in a tweet from the group after the jump! For more The Word Alive, you can purchase a concert ticket here or buy a CD…
Alternative Metal News Rock
The Word Alive Part Ways With Bassist Daniel Shapiro
August 9, 2017 Brooke Daly	alternative, announcement, Daniel Shapiro, fearless records, infectious magazine, Luke Holland, matt horn, metal, metalcore, music, news, rock, Telle Smith, the word alive, Tony Pizzuti, twa, Zack Hansen0 comment
The Word Alive have kept things pretty low key this summer after a busy stint on last year's Vans Warped Tour. Still, significant changes have taken place in the group since: ex-drummer Luke Holland left the group at the beginning of the year, and now the other half of the rhythym section will join him. The group announced bassist Daniel Shapiro has parted ways with The Word Alive. Both parties made it clear they are on good terms. Check out statements from both the band and Shapiro after the jump,…
The Word Alive Announce New Drummer
February 26, 2017 Brooke Daly	alternate edit, alternative, announcement, Daniel Shapiro, drummer, fearless records, infectious magazine, life cycles, Luke Holland, matt horn, metal, metalcore, music, new, rock, Telle Smith, the word alive, Tony Pizzuti, touring drummer, twa, Zack Hansen0 comment
In wake of their tour in Europe with Asking Alexandria, The Word Alive have announced former drummer Luke Holland's replacement. Matt Horn, who has worked with The Word Alive as a drum tech and has credits on their album Life Cycles, will be the group's touring drummer from here on out. Check out a statement from TWA welcoming Horn into the group after the jump! You can purchase a concert ticket here or buy a CD here! Please follow and like us:
Alternative Metal News
WATCH: The Word Alive, "Trapped"
March 23, 2016 March 26, 2016 Brooke Daly	"Trapped", Daniel Shapiro, dark matter, fearless records, infectious magazine, Luke Holland, metalcore, music, music video, new, premiere, Telle Smith, the word alive, Tony Pizzuti, twa, Zack Hansen0 comment
The Word Alive have had a big week. The group released a new album, the explosive Dark Matter, and now havean additional music video to accompany one of the effort's biggest tracks, "Trapped." Billboard hosted the premiere of the new clip. Equally as intense as the song itself, the music video features highly visceral shots of internal struggle interwoven with epic band performance moments. It is well worth the watch. Check it out after the jump! You can purchase a concert ticket here or buy a CD here! Please follow and…
Alternative Electronica Hardcore Metal News Pop Punk Post-hardcore
Vans Warped Tour Announce 2016 Lineup
March 22, 2016 March 22, 2016 Angie Flores	3OH!3, against the current, assuming we survive, Atreyu, Avion Roe, Bad Seed Rising, ballyhoo!, broadside, bullet for my valentine, Cane Hill, capsize, Captain Chunk!, Chelsea Grin, Chosttown, Chunk! No, coldrain, crown the empire, Cruel Hand, Dash Ten, Emarosa, every time i die, falling in reverse, four year strong, From Ashes To New, full sail university, Gideon, good charlotte, Hail the Sun, Hanna, i see stars, ice nine kills, In Hearts Wake, issues, kevin lyman, Knuckle Puck, less thank jake, like pacific, lineup, Masked Intruder, mayday parade, More To Monroe, Mother Feather, motionless in white, new found glory, oceans ate alaska, old wounds, Palaye Royale, pepper, PROF, real friends, reckless serenade, reel big fish, roam, Safe To Say, SayWeCanFly, secrets, set it off, Silent Planet, sleeping with sirens, state champs, sum 41, teenage bottlerocket, The Color Morale, The Heirs, the interrupters, the maine, The New Low, the story so far, the summer set, the word alive, tonight alive, vans, veil of maya, Volumes, wage war, warped tour, Waterparks, we the kings, whitechapel, With Confidence, yellowcard, young guns0 comment
Vans Warped Tour announced their lineup this afternoon via a live stream from Winter Park, Florida. The reveal was sponsored by Full Sail University. "I talk a lot about getting back to the basics," founder Kevin Lyman said during the lineup reveal. "The basics are the fans, the bands, and the philanthropic and educational sides of Warped Tour," he emphasized. Some of the performing acts include Mayday Parade, Emarosa, Sleeping With Sirens, and more. Check out the complete lineup, below, and let us know who you're excited to see, in the comments section. To experience…
News Rock
WATCH: Luke Holland "Where Are Ü Now" Drum Cover
April 6, 2015 Hayley Hoyle	drum cover, infectious magazine, Jack Ü, Luke Holland, music, news, rock music, the word alive, Where Are Ü Now, youtube0 comment
Luke Holland is known as the drummer of The Word Alive but he's also known for his very good drum covers of some of today's latest hits. He has released a brand new drum cover of Jack Ü's single, "Where Are Ü Now." You can check out the awesome cover after the jump! If you want to listen to his drumming skills in The Word Alive, you can purchase a CD here. If you want to catch him live on tour, you can buy concert tickets here. Please follow and like…
Metal News Rock
The Word Alive Announces Co-Headlining Tour With Chelsea Grin
March 3, 2015 Hayley Hoyle	Ashes To Ashes, Chelsea Grin, infectious magazine, like moths to flame, metal music, music, news, Real, rock music, sylar, The Real Sick Tour, the word alive, U.S. tour0 comment
The Word Alive has announced they will be heading out on a co-headlining tour with Chelsea Grin! Called The Real Sick Tour, the two bands will be making stops around the United States, and one in Canada, from the end of April through May. The Word Alive will be touring in support of their latest album, Real, while Chelsea Grin will be touring in support of their support of their album, Ashes To Ashes. Like Moths To Flames and Sylar will be joining them on tour as support. You can check out…


Students from Mayport Elementary's Coastal Sciences Academy gathered at Mandarin Park recently to celebrate the release of striped bass they raised themselves into the St. Johns River. The fifth graders raised 450 fish from hatchlings to maturity in the academy's JAXPORT-sponsored aquaculture 'Labitat'.
The outdoor lab gives students hands-on experience learning about the St. Johns River and its wildlife, as well as the river's effect on the local economy. The lab utilizes a JAXPORT-sponsored power generator that helps keep fish alive in the event of a prolonged power outage.
As an involved corporate citizen, JAXPORT sponsors and participates in numerous Northeast Florida community outreach programs focused on education and environmental stewardship.


text:
Difference between revisions of "Hamilton Piano Company"
From HtownWiki
130.217.47.57 (Talk)
(Created page with "Hamilton Piano Co. was a specialist piano store, opening c. 1945 at 361 Victoria St. on the corner of Rostrevor St. In the 1940s they advertised as sole agents for Broadwood,...")
Latest revision as of 23:08, 18 November 2019 (edit) (undo)
Thedugganaut (Talk | contribs)
Hamilton Piano Co. was a specialist piano store, opening c. 1945 at 361 Victoria St. on the corner of Rostrevor St. In the 1940s they advertised as sole agents for Broadwood, Brasted, Berry and Bell piano brands. They also offered tuition, and it was advertised that the piano tuner would visit as far afield as Bay of Plenty and Waikato.
[[image:HamiltonpianocoJuly1948.jpg|left|thumb|"Hamilton Piano Co." advertisement, 1948]]
[[image:Hamiltonpianoco1948.jpg|right|thumb|"Hamilton Piano Co." advertisement, 1948]]
Hamilton Piano Co. was a specialist piano store, which opened c. 1945 at 361 Victoria St. on the corner of Rostrevor St. In the 1940s they advertised as sole agents for Broadwood, Brasted, Berry and Bell piano brands. They also offered tuition, and it was advertised that their piano tuner was available to visit as far afield as the Bay of Plenty. The company offered free delivery with their service van.
In 1966, the company changed ownership to Koretz Investments Limited, though kept trading as Hamilton Piano Company. The store moved premises to Bryce St. at some stage. It was still in existence until approximately the early 1990s.
[[category:Musical Instrument Stores]]
"Hamilton Piano Co." advertisement, 1948
Retrieved from "http://www.expdev.net/htownwiki/index.php?title=Hamilton_Piano_Company&oldid=27421"
Musical Instrument Stores
About HtownWiki


Project Linus, a national nonprofit organization, maintains a mission of providing a sense of love, security, warmth, and comfort to children who are seriously ill or in need.
More than 7 million children in all 50 states have already received blankets. The success of Project Linus depends on the "Blanketeers," faithful volunteers who share their skills, time and talents by making these blankets.
Blankets must be new, handmade and washable, and are needed in sizes for children of all ages up to 18 years old. Blankets may be fleece, quilted, knitted or crocheted. You may use your favorite pattern or choose one from the Project Linus website, www.projectlinus.org.
Blankets are being collected through Sunday, Aug. 5. The Project Linus collection box is located on the outreach table in the main foyer of Macsherry Parish Center.


The Mitchell Law Group, LLC is a proud sponsor of the East Side Baseball Association located in Cobb County, Georgia. East Side's primary objective is to provide and maintain a high quality youth baseball program by teaching fundamentals of the game of baseball to area youths ages 5 to 18. Mr. Mitchell actively participates in this youth program as a head baseball coach.
To achieve its objectives, East Side relies on volunteerism and financial support from the community. The Mitchell Law Group is proud to sponsor and be a part of East Side.
If you would like to learn more, go to East Side Baseball Association.
Learn more about The Mitchell Law Group, please contact us.
Copyright © 2019	Mitchell Law Group	All Rights Reserved.


Feb. 17, 2013. A native of Covington, TN, Mr. Watkins was a son of the late Benjamin Franklin Watkins, Sr. and Bessie Smith Watkins. He was retired from General Motors after 37 � years in safety and production.
Survived by his wife, Blanche D. Watkins of Chapel Hill; 3 daughters, Thelma Watkins of Covington, TN; Diane Watkins of Covington, TN; Sondra Walls of Chapel Hill, TN; 3 sons, Benjamin Watkins III of Columbia, TN; Quinton Watkins of Chapel Hill, TN; Mario Walls of Chapel Hill, TN; a sister, Vickie Watkins of Covington, TN; a brother Paul Watkins of Covington, TN; 5 grandchildren; 2 great grandchildren; a host of relatives and friends.
Visitation will be 12 Noon to 2PM, Sat., Feb. 23, 2012 at Lawrence Funeral Home. A Memorial Service will be conducted Sat. at 2PM from the chapel of Lawrence Funeral Home with Pastor Marcus Duke and Deacon Benjamin Watkins III officiating.
Lawrence Funeral Home and Cremation Services, Chapel Hill, TN, 931-364-2233 (www.lawrencefuneral.net) in charge of arrangements.