(not so) Detailed Analysis of a Trade: lots of detail is lost here, you can see the problems start right at the beginning with this quote: "The 2nd column is the timestamp of the event - this is from CQS/CTA, the consolidated quote and trade SIP (Securities Information Processor)". Looks like they are determined to show only timestamps from the consolidated tape, which has been pretty much useless as a source of information for about 10 years or more. They omit the firm's receive timestamp for each message and also details of the trade execution reports received from Goldman Sigma X, which they are claiming as the source of information leakage that causes this order to be faded. Regarding Phantom Liquidity, they ask the pertinent question directly in the article: "what kind of liquidity"? but they get the wrong answer. The liquidity where you can dump a 20,000 share order directly on a market with tight spreads and not get faded does not exist now and never existed in the past; spreads were wider back in the day where you could have a hope of doing this. If you want to move large size with low slippage, you need to ship it to someone with good algorithms or spend the money to develop those algorithms yourself (along with the required technical infrastructure such as direct feeds, etc); those are really the only options available.
I don't understand why stock markets function this way. Why don't they work like a Diplomacy game, in turns?
Everyone puts their instructions in, then the turn 'executes' and an algorithm (known to all parties) resolves the actions. Run one turn every second. If you want to buy all 20k shares on offer and nobody else is making the same bid, you get them. Seems reasonably straightforward - what am I missing?
Because people who trade stock prefer it that way.
You're free to setup a market that works in this way, and if consumers prefer it you'll make millions.
I'd hazard a guess because the point of markets is to disseminate information and the point of diplomacy is to keep information secret.
Seriously, look at it like this, this buyer is a big smart human with a big brain that talks to other humans and likely knows 'secret' information about the company, lets say it's Wal-Mart and he's ordered satelite photos, etc to gauge traffic and knows they are doing well. He has all that information and all the algo knows is 'BUY 20000 @ $4.53' the algo disseminates that information to all the other traders by cancelling its quotes.
The buyer pays a premium to gain 83% of the available stock on the market. It's like Mark Cuban and his Yahoo stock, if he dumped it all the day he got it the market would tank because there's more supply than anyone wants, so he wisely instructed his broker to sell over a longer period of time. The buyer could have chose to do this, or he could attempt to buy 83% of the available stock all at once in hopes no one noticed.
The problem is that the current system is just the digital version of a bunch of guys yelling at each other on a trading floor. No one asked what the market should look like, they just took the existing system and computerized it.
Whichever order fulfills the rules of the jurisdiction the exchange resides in, I believe in the US that would mean the order that came in first.
In general orders are serialized first and then processed according to the order that was serialized, it's impossible to reason about an order book with out serializing entries to it.
It's quite easy to come up with good rules that resolve orders without serialization, and certainly without FIFO serialization. There are a number of strategies that could be employed, including "fractional allocation" and "pick randomly". I doubt it really matters exactly what you pick as long as 1) the rules are applied evenly and 2) everyone thinks they are fair.
"My ethernet cable is 5m shorter than your ethernet cable" is does not sound like a reasonable input to the fairness function.
It's easily mitigated by submitting your order 16 nanoseconds sooner.
Imagine UPS complained to the Feds that FedEx had an unfair advantage because they bought faster planes...
And actually it isn't very easy because nothing travels faster than light so you need some ridiculous method to determine with nanosecond accuracy the time that each order was 'created' at, which will invariably lead to people 'creating' orders and then not sending them.
Also the entire reason why LMAX is such a great method is because it IS serialized and IS FIFO doing those two things allows 6 million tps.
It's like seeing a car on the lot announcing I will pay $18,000 for this car and then wondering why the dealer won't sell for less than $18,000.
There's no law that says you have to trade on exchanges with HFT, you can trade on exchanges that don't allow HFT, have fun trying to find a counterparty though. Markets aren't fair they are efficient.
With enough money you can destroy practically any company. Don't like the cut of someone's jib? Buy any debt linked to stock price, short the fuck out of it then call it, leak a story about the outstanding debt and then profit on your short, buy more debt and force bankruptcy as the senior lender.
But "pick randomly" just devolves into pro-rata, which already exists in many products and has its own problems (namely that participants show more than they want). The more volume you show in this scheme, the more volume you get filled in - aka pro-rata.
I'm not sure what fractional allocation is, perhaps that's what I'm calling pro-rata? At any rate, the allocation method that is chosen is hugely important. It will have a large impact on market microstructure.
I find that the discrete mini-auctions are strictly inferior to already existing matching algorithms, as they are really just more complex (read: more gameable by HFT) , yet have the same characteristics of other, more widespread systems.
This is a fairly typical Nanex piece: start with loaded language before you've made your case, be "data-centric" yet leave out important details, intentionally exclude obvious details. By now this is all explained publicly many times over and Nanex are definitely aware of it.
In the follow-up they admit that by not being stupid you get the fill you want. Hence not rigged or stolen, just that naive execution is a poor choice: same as in unfragmented markets, same as before HFT.
Market makers put lots of quotes (orders) out on the various exchanges hoping to buy low and sell high, collecting the spread between the two. These days spreads are very small so market makers have to rely on automated quoting in order to make a profit. A side effect of this is that when markets are moving quickly the algorithms tend to take a conservative approach so the firm doesn't end up extremely long or short in a position. So when someone comes in and tries to buy up all the open orders at once they'll cancel their orders on other exchanges that haven't been executed yet so they can reprice to reflect the increased demand. The phantom liquidity really isn't much of a mystery when you look at it this way.
Because you may no longer wish to sell at that price. Similar to the way that if you have a house on the market and someone makes an offer you don't have to accept it.
It's not reasonable to assume that a human changed their mind that quickly. The idea of putting out an offer with the intention of changing your mind and letting a computer handle that process cements the point that its not a change of mind here, more likely an intentional misrepresentation of buying pressure - otherwise what's the point?
This is not what changing someone's mind looks like.
The point is to sell your shares at the highest price possible. Computers help you do that by reacting quickly to market information.
If someone sends a buy order to the exchange that quote is listed on for the same amount of shares, it will fill the entire order. What is happening here is that there are multiple exchanges and when the algo sees an order come in on one exchange it cancels orders on other exchanges quickly disseminating price information as liquidity evaporates at the current price.
The point of price is to disseminate information about the supply and demand of a good. When demand goes up price increases. The buyer is buying because he thinks the price will be higher in the future.
Both are motivated by profit, at a certain point the buyer will no longer be interested in buying at that price. The buyer obviously has a piece of information the algo doesn't and is equally trying to capitalize on the algos lack of knowledge of that information.
In the case in point the order was ~60% filled. The price the order was submitted at reflects the 'current' demand, not the demand after the order is submitted. It's pretty good to get a 60% fill when you are buying 83% of the currently available stock.
> It's not reasonable to assume that a human changed their mind that quickly
Those orders were resting there for a long time, it was not flashing.
Like you selling two adjacent houses and see a buyer purchasing one and wanting to purchase another. You understand that he/she wants the house really badly and increase the price for the second house.
Imagine you live in a city and there are currently 1000 houses on the market, you take out an ad in the news paper telling everyone that there are only 1000 and you want to buy 900 of them.
Someone wakes up at 6 AM reads the paper and phones every real estate agent in town telling them they will pay the asking price and a little bit more, most agents sell to him and he makes a nice tidy profit off of you who now have to buy from him.
Is he front running you or are you just dumb? Especially given that you could take news paper ads out all week long and then let the guy who bought 800 of the houses take the loss on them. Or you could execute a bunch of small 50 trade orders and buy all the houses you want at the asking price.
So the solution to this problem is to break up large trades then? Isn't this decreasing liquidity and if so, how big of a problem is that? Seems like trading a large amount of anything has always been challenging.
The market is already illiquid at this demand / timescale which is why the order is being partially filled, if you spread it out over 5 minutes there'd be no problem finding that much liquidity.
If this is legit, what amazes me is the time! Network hops can account for approx 100microseconds, which means that these HFT trades would not only have to be in the same datacenter, but within no more than 2-3 hops of the exchange device. Then on top of that, it's making its trade decision within 500 microseconds.
I need to sit down to calculate this, but that is guaranteed to mean that they are reading from RAM and not disk. Could the entire stock market fit in a single hashmap in memory?
That's just some awesome hardware and super efficient software.
>Could the entire stock market fit in a single hashmap in memory?
I don't see why not. You also don't have to put the entire market onto one machine - you can shard the market across multiple servers, with one server handling each symbol.
Sure, but now you've added a router, consistent hash or the like, to the picture, which is either another hop or a custom physical rack instead of a traditional ToR. It's not impossible at all, it's just impressive that they do all of this in 1ms!
>Sure, but now you've added a router, consistent hash or the like, to the picture
not necessarily. If the HFT companies are using some sort of secret internal API (very possible as they like to shave milliseconds off their execution time), it's possible that they are talking directly to relevant server, bypassing any loadbalancers or routers.
I think where Eric Hunsader (and Michael Lewis) go wrong is that they have a big flaw in their mental model of the market. They make the assumption that if say 500 shares are offered at exchange A, and 500 shares at exchange B, then they are entitled to sum those two numbers together and assume that there are 1000 shares on offer. This is wrong. This is the basic mistake that they just can't wrap their heads around. I will try to explain as simply as I can from the point of view of a market maker.
Imagine you are a market maker trying to make the spread and provide liquidity back in the old days. There is only a single stock exchange. Your firm obviously has a limit on how much risk it can take. How much capital/money it has available. So let's say you are allowed to post 1000 shares on the offer and 1000 shares on the bid. You are not allowed to hold a position of more than 1000 shares long or short. Life is simple.
Then things get complicated. A competing exchange is now set up and begins to thrive. Yay for competition! Yay for the free market! But you still have the same limits on your capital and risk. What do you do? Do you now post 500 shares on the first exchange and 500 shares on the second? You could do that. Then you are guaranteed that you will never be long or short greater than 1000 shares. You'll never break your limit. But what happens when there are 10 different stock exchanges? Do you post only 100 shares at each one? Should the available liquidity in the market be fragmented into 10 pieces? Of course not.
So what you do is post your bids and your offers for your full amount on each exchange. And then if you get filled on one exchange, you cancel your orders on the other ones. This isn't crazy, this isn't radical, this isn't some nefarious plot. It happens every day in all sorts of markets. I might put my house for sale on three different real estate websites. When it sells on one, I cancel the listing on the other two. No big deal. Or if I am a fisherman in India, I might send my sons to three different markets to sell my catch. When one finds a buyer, he phones the other two and tells them. (http://qje.oxfordjournals.org/content/122/3/879.abstract). This is simple stuff.
In more technical terms, the offers and bids are all bona fide ab initio. They are all bona fide when initiated. When posted, you do actually want to sell or buy at that price. When you get filled at one exchange, circumstances (your inventory) have now changed, and you are entitled to cancel.
Note in my explanation, the market maker isn't front-running, or even back-running, they are just responding to their own personal changes in inventory level (position) that means they can no longer offer those shares at the other exchanges. Electronic market makers may well have sophisticated models that predicts the short term price move causing them to pull orders at other exchanges, but we need not even consider those models to explain this behaviour. Changes in inventory is ample explanation.
Of course, by posting their full available inventory on each exchange, market makers expose themselves to a new risk. Someone might come along at the exact same instant and fill them on every exchange! Then they will be over their limit and massively exposed. Luckily, they have very fast systems and low latency lines to help protect them from this. If they didn't, they would be forced to offer 1/n times their current liquidity, where n is the the number of exchanges. That would suck for all liquidity consumers.
Which brings me to the most amusing part of this whole story, is that when the 'hero' of Flash Boys Brad Katsuyama figured out he could precisely fire off orders so that they hit all exchanges at the same instant in time with his 'thor' program, the SEC came knocking. Because behaviour like that, ambushing market makers on multiple exchanges, could in fact force market makers to offer much less liquidity. I don't have a problem with it, but it amuses me that the supposed hero was the one probably endangering market structure.
>"ambushing market makers on multiple exchanges, could in fact force market makers to offer much less liquidity"
The article (as well as your first paragraphs) demonstrate that this "liquidity" isn't real in the first place, and gets withdrawn if anyone tries to trade on it. So if the HFT are not offering it, nothing of value was lost!
(Indeed it is better from a macroeconomic and social standpoint, since the wasted brainpower of HFT quants goes to some other economically useful endeavour.)
> So what you do is post your bids and your offers for your full amount on each exchange
No, financial markets order books don't work like this. High-latency markets where your counterparty is a spiv who goes back on their word work like that (OTC securities, real estate or indeed dealing with people at Indian fish markets). But in an order book, orders are supposed to be firm and executable. If they aren't, the market is rigged. As it very clearly is.
>(as well as your first paragraphs) demonstrate that this "liquidity" isn't real in the first place, and gets withdrawn if anyone tries to trade on it.
Where? I have explained that it is real. The confusion stems from thinking you can sum up the orders at each exchange. It would be like seeing the same house on three different real estate websites and then thinking you can buy three houses. They are the same house. Listed three times. The liquidity listed at the 13 different US equity exchanges is listed by the same market makers, so you can't sum it. Do you understand?
>But in an order book, orders are supposed to be firm and executable. If they aren't, the market is rigged. As it very clearly is.
How aren't they executable? No one can see your order coming. If you send a single order to NASDAQ or any other exchange it will execute. They are all executable and firm at the exchange level. Nanex seems to think you should be able to execute everything visible at all exchanges at once. You can't.
> It would be like seeing the same house on three different real estate websites and then thinking you can buy three houses.
No it's not, since with the houses it would be easy to verify that there's no 3 houses available. The trader in the Nanex article had no way to know how many of the 24800 shares were actually available. He's trading at an information disadvantage.
To your analogy, it would be similar to a real estate agent flooding the market with multiple listings of each house, but with different pictures and different addresses, giving the impression of a buyers market to a buyer (phantom liquidity).
>No it's not, since with the houses it would be easy to verify that there's no 3 houses available. The trader in the Nanex article had no way to know how many of the 24800 shares were actually available. He's trading at an information disadvantage.
You are correct, public markets are anonymous by design, so there is no way to tell that the same liquidity is being offered in more than one place. That doesn't change the reality that it is. As I explained, it must be, or the transition from 1 exchange to 13 exchanges would have seen offered liquidity by market makers cut to 1/13th.
So like I said, anyone with a working mental model of the market knows this. You can't just aggregate displayed liquidity at each venue and think that number is valid. It doesn't matter what your 'screen' is telling you if the screen is doing something that makes no sense. No one ever pretended that you can magically execute on different exchanges as if they were one exchange. No one ever made that guarantee. The only people who think that are wingnuts like Eric.
And since we are talking about an issue that only effects people sending very large orders to multiple exchanges, the likes of Katsuyama have no excuses for their ignorance. They are paid to be on top of these details. You can't have a 'trader' in the modern age who doesn't know what a millisecond is, or isn't prepared or capable to dig below one abstraction layer. "dUh, my screen say 24800 shares are there and i pressed buy" is not what you pay an execution trader for. It is his fucking job to know what happens when his order goes down the wire.
>To your analogy, it would be similar to a real estate agent flooding the market with multiple listings of each house, but with different pictures and different addresses, giving the impression of a buyers market to a buyer (phantom liquidity).
They are not "flooding the market" or trying to present "phantom liquidity". They are doing exactly what you do when you put the house on 3 different websites. Trying to increase your chances of finding a customer. They are forced to put bids and offers on 13 different exchanges, because that is US equity market structure. It's no fun. Life would be a lot simpler if there were just one exchange. As I explained, it's like playing russian roulette for them... they are now forced to take the chance that they may get filled simultaneously on multiple exchanges and take on more risk than they are comfortable holding. But they are forced to do that since their customers are now spread across 13 exchanges. Market makers would like nothing better than to have 1 exchange. Arbitrageurs are the ones who love having all this fragmentation.
