[squid-users] Hyper-threading, SSD vs HDD, CentOS vs Ubuntu Server and best performer Squid version

[manuelfgarcia]

According to https://wiki.squid-cache.org/BestOsForSquid :
"CPU speed and core count:
few faster cores are better than many slow cores. SMP Squid can currently
operate most efficiently with 4-8 cores of 3GHz or more. multi-tenant
installations are better for machinery with very many cores.
only the physical cores are useful, hyper-threaded "cores" can actually be
worse.

Is that statement still up to date or is is it no longer the case from any
specific version of Squid?

Considering the statement that "hyper-threaded "cores" can actually be
worse.", what is supposed to handle faster and concurrent connections on
reverse proxy mode?:
- Intel Xeon X3430 (2.40GHz, 2,80 GHz turbo, 4 physical cores, 4 threads)
vs.
- Intel Xeon E3-1230-V6 3.40GHz (3.50 GHz, 3.90 GHz, 4 physical cores, 8
threads)

On another note, what is supposed to handle faster and more concurrent
connections on reverse proxy mode between these options?:

- SSD vs HDD

- CentOS 7 vs CentOS 6 vs Ubuntu Server 16.04 vs Ubuntu Server 18.04

- Squid 3.1 vs 3.5 vs 4.x...



--
Sent from: http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-Users-f1019091.html

[Amos Jeffries]

Post by manuelfgarcia
few faster cores are better than many slow cores. SMP Squid can currently
operate most efficiently with 4-8 cores of 3GHz or more. multi-tenant
installations are better for machinery with very many cores.
only the physical cores are useful, hyper-threaded "cores" can actually be
worse.
Is that statement still up to date or is is it no longer the case from any
specific version of Squid?

It is still true and correct.

Post by manuelfgarcia
Considering the statement that "hyper-threaded "cores" can actually be
worse.", what is supposed to handle faster and concurrent connections on
reverse proxy mode?

Reverse-proxy mode gains its performance benefits through only having to
service a limited range of URLs. Which means more predictability and
objects do not get evicted from memory cache quite as often as from
other proxies. Memory is faster than disk or network. Nothing to do with
CPU(s).

Post by manuelfgarcia
- Intel Xeon X3430 (2.40GHz, 2,80 GHz turbo, 4 physical cores, 4 threads)
vs.
- Intel Xeon E3-1230-V6 3.40GHz (3.50 GHz, 3.90 GHz, 4 physical cores, 8
threads)
On another note, what is supposed to handle faster and more concurrent
- SSD vs HDD

These are disk drives. Not networks. Disks have nothing to do with
"concurrent connections". Squid can handle as many concurrent
connections as your servers TCP stack allows it to. No more, no less.


Squid performs random-access I/O to on-disk caches. With an order of
magnitude or so higher than normal write accesses. These non-normal I/O
patterns prevent typical disk controllers I/O prediction models from
working efficiently.

So be aware that whichever disk type you choose will have its lifetime
reduced to some fraction of what any manufacturer specs' it as being
capable of. How much depend on your specific traffic.

SSD are still faster hardware than HDD. But also typically wear out
faster and have smaller content capacity. So it is a matter of whether
you can afford the budget to replace SSD or HDD as they wear out. And
you can easily change between them at any time if you want to experiment.

Post by manuelfgarcia
- CentOS 7 vs CentOS 6 vs Ubuntu Server 16.04 vs Ubuntu Server 18.04
- Squid 3.1 vs 3.5 vs 4.x...

Squid-3.x are officially deprecated by Squid-4 being published. Similar
deprecation situations apply for the OS you mention.

A good rule of thumb for any new installation is to start with the
latest tech you are able. It will have the longest support / warranty
period as measured from the day you decide to put it into service. If
you start with tech that is old, you are cutting your support timespan
by that much.

Another (possibly conflicting) rule of thumb, is to reduce the amount of
change you are going to have to learn. If you are already familiar with
a particular OS go with that while you learn how to operate the Squid
part of the system. Squid operates approximately as well in any OS -
excluding Windows, which sadly imposes a few severe network limitations.


HTH
Amos

[Matus UHLAR - fantomas]

Post by manuelfgarcia
only the physical cores are useful, hyper-threaded "cores" can actually be
worse.

HT can add about 10-15% of CPU time when doing CPU inteensive tasks.
squid usually contains no CPU intensive tasks.

Post by manuelfgarcia
Considering the statement that "hyper-threaded "cores" can actually be
worse.", what is supposed to handle faster and concurrent connections on
- Intel Xeon X3430 (2.40GHz, 2,80 GHz turbo, 4 physical cores, 4 threads)
vs.
- Intel Xeon E3-1230-V6 3.40GHz (3.50 GHz, 3.90 GHz, 4 physical cores, 8
threads)

I would count both of them as 4-CPU machines, e.g. would not configure more
than four (apparently only three) workers for squid.

Luckily kernels seem to understand hyperthreading CPUs and don't use HT
unless needed, so the work will be spread over 4 cores in such case.

[Alex Rousskov]

Post by manuelfgarcia
few faster cores are better than many slow cores. SMP Squid can currently
operate most efficiently with 4-8 cores of 3GHz or more. multi-tenant
installations are better for machinery with very many cores.
only the physical cores are useful, hyper-threaded "cores" can actually be
worse.
Is that statement still up to date or is is it no longer the case from any
specific version of Squid?

There are several statements in "that statement":

* The statement about fewer faster cores being better can be true or
false for SMP Squids, depending on how much fewer and how much faster
those cores are. That statement should be removed or toned down IMO.

* The statement that only the physical cores are useful is bogus.
Virtual cores can be useful in some environments. For busy systems with
many cores, it is usually a bad idea to put multiple busy processes
(e.g., Squid workers) on one physical core, but that does not mean that
ignoring or disabling virtual cores is always a good idea. The optimal
process:core mapping depends on many factors.

* The statement about hyperthreading sometimes leading to worse
performance is still correct. Improper use of hyperthreading can
decrease performance compared to disabling hyperthreading. However, that
does not mean one should always disable hyperthreading.

The above applies to all modern Squids.

Post by manuelfgarcia
Considering the statement that "hyper-threaded "cores" can actually be
worse.", what is supposed to handle faster and concurrent connections on
- Intel Xeon X3430 (2.40GHz, 2,80 GHz turbo, 4 physical cores, 4 threads)
vs.
- Intel Xeon E3-1230-V6 3.40GHz (3.50 GHz, 3.90 GHz, 4 physical cores, 8
threads)

In most cases, it is not so much the _presence_ of virtual cores that
matters. It is whether you overload physical cores by placing too many
busy processes on them. Thus, if each of the two machines is correctly
configured/optimized, then I would bet on E3-1230-V6 because it has the
same number of physical cores but they are faster.

Post by manuelfgarcia
On another note, what is supposed to handle faster and more concurrent
- SSD vs HDD

SSD if disk I/O is a bottleneck.

Post by manuelfgarcia
- CentOS 7 vs CentOS 6 vs Ubuntu Server 16.04 vs Ubuntu Server 18.04

I would not expect a huge difference if each is properly configured and
tuned, but we have not compared these four in our lab.

Post by manuelfgarcia
- Squid 3.1 vs 3.5 vs 4.x...

Squid v2 :-). If you are selecting a Squid version based primarily on
its speed, you will probably regret your choice. My suggestion is to
avoid Squid v3.1 in all cases and use Squid v4 if you can.


HTH,

Alex.